update-adopt CLI honesty: no-candidate, a status that admits it cannot read root state, and the --no-check-sigs note

docs/README.md

@@ -24,4 +24,5 @@ it.
 | [`util-01-sampler-reconcile.md`](util-01-sampler-reconcile.md) | **How the two-commit UTIL-01 sampler branch reached `main` with its semantics unchanged** — the one overlapping import line, the digests the card locks, the two `[E]` probe rows that ship RED on purpose, and the U-D17 oracle. |
 | [`old/`](old/) | Archival stub: an index of the retired documentation set and how to read it back from Git history. |
 | [`local-ai/wanted-packages/`](local-ai/wanted-packages/) | Findings the fleet turned up but has not acted on — one page per finding, each naming what stopped, what is measured, and what is inference. |
+| [`fleet-candidate-adoption.md`](fleet-candidate-adoption.md) | **Publishing a candidate the NAS did not build, and why that `nix copy` needs `--no-check-sigs`** — where the trust in that transfer actually comes from, the three places the flag is never legitimate, and how to read `update-adopt status` (why it exits 1 as tom, and `no-candidate` against `fetch-failed`). |
 | [`runbook-conventions.md`](runbook-conventions.md) | **How a runbook suppresses a unit on NixOS** — why a plain mask fails on the read-only `/etc/systemd/system` store symlink and `mask --runtime` silently does nothing, and the intent → wrong → right table (runtime drop-in, `enable = false` + rebuild, move the data, stop the timer). |

docs/fleet-candidate-adoption.md

@@ -0,0 +1,100 @@
+# Publishing a fleet candidate by hand, and reading an endpoint's answer
+
+Written 2026-09-16 for #354 (the four items left on its 2026-09-14 status
+comment). The nightly path needs no operator: `update-center` on the NAS
+resolves `main` once, builds each enrolled host, pushes to Attic and publishes
+a signed per-host manifest, and each endpoint's `update-adopt` timers stage and
+(on the rolling policy) activate it. This page is for the two times an operator
+is in the loop — publishing a closure built somewhere else, and reading
+`update-adopt status` — and for the one flag in that path that looks like a
+safety bypass and is not.
+
+The Nix files that decide the behaviour here are `hosts/nas/update-center.nix`
+(with its body in `hosts/nas/update-center.sh`) and `modules/update-adopt.nix`
+(body in `modules/update-adopt.py`).
+
+## Publishing a closure the NAS did not build
+
+The NAS publishes only what is already in its own store and already in Attic:
+`update-center --publish-only HOST STORE_PATH FLAKEREF` refuses a path that is
+not valid locally, and publishes no pointer unless `attic push` succeeds
+first. So a closure built on the coordinator has to be copied to the NAS first:
+
+```sh
+# on the coordinator, as tom
+nix copy --to ssh-ng://root@nas --no-check-sigs /nix/store/<hash>-nixos-system-worker-<ver>
+# on the NAS, as root
+update-center --publish-only worker /nix/store/<hash>-nixos-system-worker-<ver> \
+  github:mecattaf/dotfiles/<40-char rev>
+```
+
+### Why `--no-check-sigs` belongs on that line
+
+A path you just built is unsigned. Nothing signs a local build: signatures are
+added by a cache when it serves the path, so `attic push` signs with the
+`fleet:` key on the way into the cache and nothing signs it on the way into the
+NAS's store. The receiving daemon enforces `require-sigs` unless the client
+asks it not to, so the copy above fails with *"cannot add path … because it
+lacks a signature by a trusted key"* until `--no-check-sigs` is passed.
+
+The flag is honoured only for a trusted user — `trusted-users = [ "root"
+"@wheel" ]` in `modules/common.nix` — which is what makes it work as root to
+the NAS and what makes it useless to anyone else.
+
+It is legitimate here for one reason: **the trust in this transfer comes from
+the transport and from who built the bytes, not from a signature.** You built
+the closure on a fleet host, and you are handing it over an authenticated SSH
+channel to a store you administer. A signature would only re-prove a fact you
+already have.
+
+It is **not** legitimate anywhere the signature is the only evidence:
+
+- never on an endpoint substituting from a cache. `http://nas:8080/fleet` is
+  trusted by its public key (`extra-trusted-public-keys` in
+  `modules/common.nix`), and that key is the whole proof that the bytes are the
+  ones the NAS built;
+- never to silence a signature error whose cause you have not found. An
+  unsigned path from a cache is a broken cache or a wrong key, and copying it
+  anyway hides that;
+- never in a unit or a timer, and never as a habit on `nix copy`. The seed
+  timer in `home/update-center-seed.nix` copies the lock's private source trees
+  to the NAS nightly and does *not* pass it: those paths are narHash-addressed,
+  so their own hash is the proof and no signature is wanted. A path that needs
+  the flag is a path whose only possible proof is who you are.
+
+Adoption's own trust is separate from all of this and is not weakened by the
+flag: the endpoint verifies the manifest's `ssh-keygen -Y` signature in the
+`fleet-update` namespace against an allowed-signers file holding the NAS's host
+key from `modules/mesh-registry.nix` (rendered into the store by
+`modules/update-adopt.nix`), checks
+that `store_path` names *this* host's system closure, and re-reads the
+revision from the closure's own `fleet-revision.json` rather than trusting the
+manifest's copy of it.
+
+## Reading `update-adopt status`
+
+Two answers that used to read as "fine" now say what they are.
+
+**`status` as tom exits 1.** The state directory is `/var/lib/update-adopt`,
+mode 0700 root. A run that cannot read it prints the facts it can still see
+(`current`, `booted`, `profile` and the running generation's revision), sets
+`state_unreadable` to the reason, leaves `state`, `candidate`, `last_refusal`,
+`last_known_good` and `pending_reboot` at `null`, and exits 1. It no longer
+prints `"state": "idle"` for a host it never looked at. Run it under `sudo`.
+`fleet-status` collects every host, the coordinator included, as root over ssh
+for exactly this reason (`modules/fleet-status.nix`); if it ever runs the verb
+as a non-root user it grades the answer `unknown`, not "up to date".
+
+**A 404 manifest is `no-candidate`, not `fetch-failed`.** Both still exit 0 —
+neither is this host's failure — but they are different facts and the journal
+now names them differently:
+
+| Journal line | What happened | What to do |
+|---|---|---|
+| `reason=no-candidate` | The NAS answered, with 404: it has published nothing for this host. Normal for a host that is not enrolled, and for any host before the first successful nightly | Check `update-center` on the NAS, not the endpoint |
+| `reason=fetch-failed` | No HTTP answer at all: the NAS is down, or off the mesh from here | Check reachability |
+
+Read them with
+`journalctl -t update-adopt -o cat` (the identifier is set by the script, so a
+unit glob is not needed), or from the last refusal in `sudo update-adopt
+status --json`.

flake.nix

@@ -2223,7 +2223,10 @@
         # unknown gates, probe-failure rollback, switch-failure rollback,
         # kernel → boot + pending-reboot marker, bad signature, host mismatch,
         # manual and stage-only policies, downgrade refusal before and after
-        # download, dirty/unknown current, --force, the activation lock) and
+        # download, dirty/unknown current, --force, the activation lock, a 404
+        # manifest as `no-candidate` against an unreachable NAS as
+        # `fetch-failed`, and a `status` that cannot read the root-only state
+        # exiting 1 rather than printing an empty one) and
         # each gate verb of update-adopt-gates.sh, with fake nix/systemctl and a
         # REAL ssh-keygen. The asserts pin the policy table: NAS not enrolled
         # (2026-08-21 ruling), client manual (R-18), both units immune to the

modules/update-adopt.py

@@ -8,9 +8,13 @@
 status    print the freshness facts as JSON
 
 Exit codes are a contract with failure-surfacing: a busy host, a newer local
-generation, a manual policy or an unreachable NAS exit 0 (a receipt, not a
-failure); a bad signature, a failed realise, or a failed probe (after the
-local rollback) exit 1, so the unit fails and a marker is written.
+generation, a manual policy, an unreachable NAS or a host the NAS has not
+published a candidate for exit 0 (a receipt, not a failure); a bad signature,
+a failed realise, or a failed probe (after the local rollback) exit 1, so the
+unit fails and a marker is written. `status` exits 1 when it cannot read the
+state it is reporting on — a run as tom against the root-only state directory
+prints what /run and /nix show and says the rest is unknown, instead of an
+empty state that reads like a healthy idle host.
 
 Test seams (all unset in the real unit; see modules/update-adopt.nix):
   UPDATE_ADOPT_CONFIG     JSON config rendered by Nix (required)
@@ -110,6 +114,27 @@ def journal(state, reason, **fields):
         pass
 
 
+def state_unreadable():
+    """Why this user cannot read the host's adoption state, or None.
+
+    /var/lib/update-adopt is 0700 root, so `update-adopt status` run as tom
+    opens neither the directory nor state.json — and load_state() would hand
+    back an empty dict that prints exactly like a healthy idle host. Nothing
+    staged YET is a real answer (no state file on a directory this user can
+    read, or no directory at all on a host that never ran stage); state this
+    user is not allowed to see is not.
+    """
+    try:
+        with open(os.path.join(STATE_DIR, "state.json")):
+            return None
+    except FileNotFoundError:
+        if os.path.isdir(STATE_DIR) and not os.access(STATE_DIR, os.R_OK | os.X_OK):
+            return f"{STATE_DIR} is not readable by uid {os.geteuid()}"
+        return None
+    except OSError as error:
+        return str(error)
+
+
 def load_state():
     try:
         with open(os.path.join(STATE_DIR, "state.json")) as stream:
@@ -261,8 +286,18 @@ def fetch_candidate(cfg, workdir):
     manifest_path = os.path.join(workdir, "manifest.json")
     sig_path = manifest_path + ".sig"
     for url, dest in ((base + "/manifest.json", manifest_path), (base + "/manifest.json.sig", sig_path)):
-        result = run(["curl", "-fsS", "-m", "10", "-o", dest, url], timeout=30)
+        # -w prints the status line's code (000 when no HTTP answer arrived),
+        # so "the NAS published nothing for this host" is told apart from "the
+        # NAS did not answer". Both exit 0; only their names differ, and the
+        # name is what an operator reads in the journal.
+        result = run(["curl", "-fsS", "-m", "10", "-w", "%{http_code}", "-o", dest, url], timeout=30)
         if result.returncode != 0:
+            if result.stdout.strip() == "404":
+                raise Refusal(
+                    "no-candidate",
+                    f"{url}: HTTP 404 — the NAS has published no candidate for this host",
+                    rc=0,
+                )
             raise Refusal("fetch-failed", f"{url}: {result.stderr.strip()}"[:300], rc=0)
     with open(manifest_path, "rb") as stream:
         raw = stream.read()
@@ -753,9 +788,13 @@ def _activate(cfg, force):
 
 
 def status(cfg):
-    state = load_state()
+    unreadable = state_unreadable()
+    # Report no state at all rather than load_state()'s defaults: an "idle"
+    # this process never read is a claim, not a fact.
+    state = {} if unreadable else load_state()
     current = resolve(CURRENT)
     out = {
+        "state_unreadable": unreadable,
         "host": cfg["host"],
         "policy": cfg["policy"],
         "state": state.get("state"),
@@ -774,6 +813,15 @@ def status(cfg):
     }
     json.dump(out, sys.stdout, indent=2, sort_keys=True)
     sys.stdout.write("\n")
+    if unreadable:
+        print(
+            f"update-adopt: {unreadable}; the state, candidate, refusal and "
+            "known-good fields above are unknown, not empty. Run it as root "
+            "(`sudo update-adopt status --json`); fleet-status collects it "
+            "over ssh as root for this reason.",
+            file=sys.stderr,
+        )
+        return 1
     return 0
 
 

tests/update-adopt/test_update_adopt.py

@@ -22,14 +22,27 @@
 GATES = os.path.join(REPO, "modules", "update-adopt-gates.sh")
 
 FAKES = {
+    # Models the two failures that look alike from the outside: a NAS that
+    # answers 404 (nothing published for this host) and a NAS that does not
+    # answer at all (curl exit 7, no status line, %{http_code} = 000).
     "curl": r"""#!/bin/sh
-dest=""; url=""
+dest=""; url=""; code=""
 while [ $# -gt 0 ]; do
-  case "$1" in -o) dest="$2"; shift 2 ;; -m) shift 2 ;; -*) shift ;; *) url="$1"; shift ;; esac
+  case "$1" in -o) dest="$2"; shift 2 ;; -m|-w) shift 2 ;; -*) shift ;; *) url="$1"; shift ;; esac
 done
+if [ -e "$FIXTURE/nas-down" ]; then
+  printf 000
+  echo "curl: (7) Failed to connect to nas.test port 8734" >&2
+  exit 7
+fi
 src="$FIXTURE/www/${url#http://nas.test/}"
-[ -f "$src" ] || exit 22
+if [ ! -f "$src" ]; then
+  printf 404
+  echo "curl: (22) The requested URL returned error: 404" >&2
+  exit 22
+fi
 if [ -n "$dest" ]; then cp "$src" "$dest"; else cat "$src"; fi
+printf 200
 """,
     "nix-store": r"""#!/bin/sh
 echo "nix-store $*" >> "$FIXTURE/calls.log"
@@ -475,8 +488,30 @@ def test_host_mismatch_refused(self):
         self.assertNotIn("--realise", self.fx.calls())
 
     def test_unreachable_nas_is_not_a_failure(self):
-        self.assertOk(self.fx.run("stage"))
+        open(os.path.join(self.fx.tmp, "nas-down"), "w").close()
+        result = self.fx.run("stage")
+        self.assertOk(result)
         self.assertEqual(self.fx.state()["last_refusal"]["reason"], "fetch-failed")
+        self.assertIn("reason=fetch-failed", result.stdout)
+
+    # A NAS that answers 404 has published nothing for this host. That is not
+    # a fetch that failed, and the journal must not say it was (#354 item 4).
+    def test_missing_manifest_is_no_candidate(self):
+        result = self.fx.run("stage")
+        self.assertOk(result)
+        self.assertIn("reason=no-candidate", result.stdout)
+        refusal = self.fx.state()["last_refusal"]
+        self.assertEqual(refusal["reason"], "no-candidate")
+        self.assertIn("404", refusal["detail"])
+        self.assertEqual(self.fx.receipts()[-1]["reason"], "no-candidate")
+        self.assertNotIn("--realise", self.fx.calls())
+        # And it is still a 404 once the host HAS a candidate that is unpublished
+        # again: the honest name does not depend on prior state.
+        self.fx.publish(self.fx.system("b", last_modified=2000))
+        self.assertOk(self.fx.run("stage"))
+        os.unlink(os.path.join(self.fx.tmp, "www/candidates/worker/manifest.json"))
+        self.assertOk(self.fx.run("stage"))
+        self.assertEqual(self.fx.state()["last_refusal"]["reason"], "no-candidate")
 
     # (6) manual policy → never realises
     def test_manual_policy_only_reports(self):
@@ -569,6 +604,34 @@ def test_status_reports_freshness(self):
             self.assertIn(key, out)
         self.assertEqual(out["candidate"]["store_path"], b)
         self.assertEqual(out["current_revision"]["lastModified"], 1000)
+        self.assertIsNone(out["state_unreadable"])
+
+    # The real state directory is 0700 root; a run that cannot read it must
+    # not print an empty state that looks like a healthy idle host (#354).
+    @unittest.skipIf(os.geteuid() == 0, "root reads a 0700 directory anyway")
+    def test_status_without_access_to_state_exits_nonzero(self):
+        self.fx.publish(self.fx.system("b", last_modified=2000))
+        self.assertOk(self.fx.run("stage"))
+        os.chmod(self.fx.state_dir, 0o000)
+        try:
+            result = self.fx.run("status", "--json")
+        finally:
+            os.chmod(self.fx.state_dir, 0o700)
+        self.assertEqual(result.returncode, 1, result.stdout + result.stderr)
+        out = json.loads(result.stdout)
+        self.assertIsNotNone(out["state_unreadable"])
+        for key in ("state", "candidate", "last_refusal", "last_known_good", "pending_reboot"):
+            self.assertIsNone(out[key], key)
+        # The facts it CAN read are still reported, and still true.
+        self.assertEqual(out["current"], self.fx.a)
+        self.assertIn("unknown, not empty", result.stderr)
+
+    def test_status_with_no_state_yet_is_not_an_error(self):
+        result = self.fx.run("status", "--json")
+        self.assertOk(result)
+        out = json.loads(result.stdout)
+        self.assertIsNone(out["state_unreadable"])
+        self.assertEqual(out["state"], "idle")
 
     def test_receipts_are_capped(self):
         for _ in range(60):