Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions docs/guides/hetzner_deployment_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -865,11 +865,15 @@ fixed order:
diverge a little further after **every** release, and
`git log main..development` starts counting work that already shipped.

5. **Deploy** = three steps on the server, not one: the tag file, then
`git fetch --tags && git checkout <tag>`, then `uv sync`. Writing the
tag file alone is **not** deploying — that mistake left the host on
v1.10.0 for five days while every signal read v1.11.0 (C-343).
`docs/guides/server_quickref.md` holds the commands.
5. **Deploy** = four blocks on the server, not one: the tag file, then
`git fetch --tags && git checkout <tag>`, then `uv sync`, then the
verification that the tag file, the checked-out tag and the installed
distribution all name the same version. Writing the tag file alone is
**not** deploying — that mistake left the host on v1.10.0 for five days
while every signal read v1.11.0 (C-343), and the fourth block is what
would have caught it. `docs/guides/server_quickref.md` holds the
commands; if this count and the quickref ever disagree, the quickref
wins.

**Why this order:** If you tag before bumping, the tag points at code
with the old version string — wrong. If you tag on `development`
Expand Down
37 changes: 35 additions & 2 deletions reports/technical_risk_register.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@
| C-340 | 3 | Auto-merge fails silently two ways: `gh pr merge --auto --<method>` refuses to change the method on an already-armed PR (nearly put a squash on `main`), and pushing to a branch whose PR already merged orphans the commit with no error | **Before arming a non-default merge method**, read `auto_merge.merge_method` back; **before any follow-up push to a PR branch**, check `merged` first or use a new branch | Operational safety |
| ~~C-341~~ | ~~4~~ | ~~Deploy gates only run where someone types pytest — C-320's fix made them skip-with-reason in CI, so they assure only whoever runs the suite at the right moment | **When adding a deploy gate, or relying on one for release assurance:** ~~ | Resolved 2026-08-11 (#424 gave the answerable gates a scheduled runner; #425 deleted the one that could never answer and replaced it with a publish-time guard) | Test infra |
| ~~C-342~~ | ~~3~~ | ~~A stale committed `uv.lock` is invisible — `uv sync` (ci.yml:24,42,60,99) rewrites it in place, so CI goes green on a lock that does not match the committed `pyproject.toml` and the stale one stays in git~~ | Resolved 2026-08-11 (#424): `uv lock --check` in the `test` job, before `uv sync`; drilled both directions and the ordering is itself guarded by `tests/test_ci_gates.py` | Dependency policy |
| C-343 | 2 | Writing `~/.views-deploy-tag` is not deploying — the server ran v1.10.0 for five days while the tag file said v1.11.0 and views-frames stayed at the frozen 1.0.0. The in-script `git checkout` cannot fix the running script (bash has buffered it) and never runs `uv sync` | **At the next release** — deploy with all three `server_quickref.md` steps, then verify tag file, `git describe --tags` and installed version all agree | Deployment |
| C-343 | 2 | Writing `~/.views-deploy-tag` is not deploying — the server ran v1.10.0 for five days while the tag file said v1.11.0 and views-frames stayed at the frozen 1.0.0. The in-script `git checkout` cannot fix the running script (bash has buffered it) and never runs `uv sync` | **At the next release** — deploy with all four `server_quickref.md` blocks, the fourth being the verification that tag file, `git describe --tags` and installed distribution all name the same version | Deployment |
| C-347 | 3 | The required CI check exercises a different raster decoder than production — since the 3.11 floor the lock forks, so `test` decodes with `imagecodecs 2026.3.6` while the server uses `2026.5.10`. `test-py313` covers the production line but is **not a required check**, and a red nobody must satisfy is ignorable | **Fired and actioned 2026-08-13.** `test-py313` reported green on `development` and on `main`, and was then added to the required-status-check lists on both (readback verified). Re-opens if the job is renamed or the requirement dropped | Test infra |
| C-348 | 3 | Nothing asserts which Python the production server runs, and the floor now admits one that installs a **different raster line**. `preflight.py`, `check_health.py` and `refresh_pipeline.sh` contain no `sys.version_info` check of any kind. Created by #443 — this risk did not exist under `>=3.12`. **Measured 2026-08-13: the host runs Python 3.12.3 (Ubuntu 24.04's system interpreter), so it resolves `tifffile 2026.5.15` / `imagecodecs 2026.5.10` — the current fork.** Reassuring, and it is an accident: the guide said "Install Python 3.10+" and this distro happened to hand it 3.12. Still OPEN because one observation is not an assertion | **At the next server provisioning, Python upgrade on the Hetzner host, or any runbook edit that says `apt install python3`** — pin the interpreter explicitly and record which raster fork it resolves | Server hardening |
| C-349 | 4 | A config value restated in prose has nothing binding it back — `hetzner_deployment_guide.md` said "Install Python 3.10+" for the three months `pyproject.toml` declared `>=3.12`, an instruction producing an environment where the package could not install. The #444 pin guard binds *workflow* pins to `requires-python`; nothing binds *prose* | **When writing a Python version, or any pyproject value, into a guide or ADR** — link to the declaration instead of restating it, or accept that the copy will not be checked | Documentation drift |
Expand Down Expand Up @@ -3199,7 +3199,7 @@ Cross-ref: C-337 (the floor-versus-lock concern this sits beside — same file,

**Source:** direct observation on the production host, 2026-08-08, during epic #421. Not a code reading — the operator ran the commands and pasted the output.

**Trigger:** **At the next release.** Deploy with all three steps from `server_quickref.md`, then verify the three sources agree — tag file, `git describe --tags`, installed version. A release that changes `scripts/refresh_pipeline.sh` or any dependency is the acute case.
**Trigger:** **At the next release.** Deploy with all four blocks from `server_quickref.md`; the fourth is the verification that the three sources agree — tag file, `git describe --tags`, installed distribution. A release that changes `scripts/refresh_pipeline.sh` or any dependency is the acute case.

**Location:** `scripts/refresh_pipeline.sh` (deployment-gate header comment, and the `git checkout "$DEPLOY_TAG"` in the deployment gate), `docs/guides/server_quickref.md` §"Deploy a new version".

Expand Down Expand Up @@ -3309,6 +3309,39 @@ Corrected by kind rather than uniformly, which is the part worth keeping: the fo

**What this update changes about the entry's severity, honestly.** It does not raise the tier, but it does weaken the "corrected in the same PR as this entry" claim above: the correction was partial, and the stale copies outlived it by two months and one release. The pattern is #402's — *one procedure, one home* — and C-343 is an instance of it as much as it is a deployment risk. The entry stays **open**, and the still-unbuilt instrument below is now the only thing that would catch a sixth copy appearing.

**Update 2026-10-20 — the corrected procedure completed, step 4 included, for the first time.** v1.15.0 deployed with all four blocks and all three verification commands agreeing:

```
cat ~/.views-deploy-tag → v1.15.0
git describe --tags --exact-match → v1.15.0
ls .venv/lib/python*/site-packages | grep views_data… → views_datafactory-1.15.0.dist-info
```

`uv sync` again printed the transition — `- views-datafactory==1.14.0` / `+ views-datafactory==1.15.0` — so block 3 is load-bearing twice measured, not once. **What is new is that step 4 ran at all:** on 2026-10-16 its first command died on the `~`-expansion defect, so the 2026-10-13 verification block had never once been executed successfully. The #498 fix is now confirmed by execution rather than by reading, which is the distinction this entry keeps failing on.

**What this deploy did *not* test, stated so the confirmation is not read as broader than it is.** `scripts/refresh_pipeline.sh` is byte-identical between v1.14.0 and v1.15.0, so the buffered-running-script half of this entry — the reason a `refresh_pipeline.sh` change lands one cron later — was not exercised. Two of the three mechanisms are now confirmed working; the third remains confirmed only by the August incident that created the entry. And nothing yet *refuses* a mismatched deploy: step 4 detects, it does not gate.

**A fifth instance, one layer up, by the author of the fourth correction.** During this release the instrumentation for #501 was merged to `development` and then described — in session, to the operator — as something the next cron run would pick up "automatically". It would not have: the server runs a tag, and `v1.14.0` did not contain it. *Merging the instrumentation is not deploying the instrumentation* is the same confusion as *writing the tag is not deploying*, displaced from the tag file to the branch, and it was committed three days after this entry's fourth round by whoever wrote that round. Corrected on #501 before any wait was incurred. The same slip recurred twice more in the same session as chat-only paraphrase ("three steps" for the four blocks), each time from recalling the procedure instead of reading it — not registrable, since chat is not a repo artifact, but the frequency is the evidence: five instances across three months, every one from *reasoning about* the procedure rather than *opening* it.

**Four more copies still said "three steps", and two of them were in this entry.** Writing the update above prompted a grep for the step *count* rather than the step *content* — a dimension none of the four previous rounds had audited. All four were live, and the correction is round five:

| Location | Said |
|---|---|
| `reports/technical_risk_register.md` — C-343's row in `## Open Items Summary` | *"deploy with all three `server_quickref.md` steps"*, then listed the three checks that constitute step **4** — four things while naming three |
| `reports/technical_risk_register.md` — C-343's own **Trigger:** field | *"Deploy with all three steps from `server_quickref.md`"* |
| `docs/guides/hetzner_deployment_guide.md` §"the release ritual" item 5 | *"**Deploy** = three steps on the server, not one"*, enumerating exactly three |
| `scripts/refresh_pipeline.sh` header comment | *"with ALL THREE steps in docs/guides/server_quickref.md"* |

The last is the sharpest: that comment's very next sentence says *"two copies of a procedure drifting apart is what C-343 IS, and a fix that adds a third copy would be the same bug"* — and it had drifted, because **a count is a copy too.** The comment carefully avoided duplicating the commands and then duplicated the number, which was the one part that changed.

Not registrable as a new entry: same entry, same mechanism, fifth round. But it relocates where the mechanism hides. Rounds one through four hunted copies of the *procedure* and found them in guides and ADRs; this round found that the procedure's **cardinality** is independently copyable, and that two of the four stale copies were inside the register entry doing the hunting — including its trigger, the field an operator is meant to act on. #500 audited seven duration figures across the guides and did not check a step count, because nothing suggested a count could rot.

**Deliberately left as written**, because each is a dated record rather than an instruction, and per #500's precedent a record is amended and never rewritten to look current: ~~C-331~~'s narrative at *"deploying is three steps, not one"* (true when written, and the sentence is about the one-run lag, not the procedure); ADR-022's *"This section previously gave deploying as three steps"* (correct as a description of what it previously said); and `reports/register_changelog.md`'s account of a past redeploy *"run as all three `server_quickref.md` steps"* (it was three at the time).

`docs/guides/server_quickref.md:136` says *"use all three checks in §'Deploy a new version' step 4"* and is **correct** — step 4 does contain three checks. Four blocks, the fourth holding three commands, is what makes "three" and "four" both appear legitimately in the same procedure, and is a fair part of why the count drifted.

**What that implies for the instrument below, which is now the entry's only remaining work.** Four rounds of documentation correction have produced a procedure that executes correctly when followed. They have not produced anything that notices when it is not followed, and the failure mode has simply moved — from stale copies, to a stale mental model of where code must be before it runs. A pre-flight gate cannot catch the #501 variant either. What would is the duration/version comparison noted as absent in #502's review: nothing compares what the server *is running* against what the repository *expects*, at any layer.

**The instrument, not yet built.** A pre-flight check in `refresh_pipeline.sh` comparing `$DEPLOY_TAG` against the HEAD the script *started* from, and refusing to run when they disagree — fail-loud per ADR-011, rather than silently self-correcting one run late. Proposed for #424; deliberately not built here, because this PR is a register entry and a comment fix.

Cross-ref: C-337 (the fix this failed to deliver), C-342 (the other lockfile-versus-reality entry from the same day), C-341 (gates that run nowhere), C-98/ADR-022 (the deployment gate this undermines). Part of the **mechanisms that fail green** cluster. GitHub: #421.
Expand Down
8 changes: 6 additions & 2 deletions scripts/refresh_pipeline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,15 @@
# 2. `uv sync` never runs, so a dependency change never lands at
# all, no matter how many times the pipeline runs.
#
# Deploy — and roll back — with ALL THREE steps in
# Deploy — and roll back — with ALL FOUR BLOCKS in
# docs/guides/server_quickref.md §"Deploy a new version": the tag
# file, then the git fetch/checkout, then `uv sync`. Deliberately not
# file, then the git fetch/checkout, then `uv sync`, then the
# verification that all three name the same version. Deliberately not
# repeated here: two copies of a procedure drifting apart is what
# C-343 IS, and a fix that adds a third copy would be the same bug.
# This comment said THREE until 2026-10-20, five weeks after the
# fourth block was added — a count is a copy too, which is exactly
# the drift the sentence above warns about.
# **If this comment and the quickref ever disagree, the quickref
# wins.** See docs/guides/hetzner_deployment_guide.md for full detail.
#
Expand Down
Loading