20260922 - Adopt node ingest v1.4.0, and claim a node by email

CLAUDE.md

@@ -4,7 +4,7 @@ The node-side telemetry uplink for the RETINA passive radar fleet. One container
 node, owning everything sent to the server: registration, detection streaming,
 heartbeat, config sync. Nothing else on the node talks to `api.retina.fm`.
 
-**Status: built, and implementing spec v1.2.2.** Verified end to end on the Owl node
+**Status: built, and implementing spec v1.4.0.** Verified end to end on the Owl node
 against a tunnelled mock — every endpoint, every reachable state including `stalled`,
 and the refusal paths.
 
@@ -50,12 +50,13 @@ Corollary: **all unit conversion happens in stage 2.** Stage 1 hands over source
 under names that say so — `delay_km`, `timestamp_ms`, `rx_alt_m` — and stage 2 emits the
 spec's names and units. A missing conversion is then visible at the call site.
 
-## Three files retina-gui writes, two of which gate registration
+## Four files retina-gui writes, two of which gate registration
 
-All three live under `/data/retina-gui`, mounted read-only, and nothing in any of them
+All four live under `/data/retina-gui`, mounted read-only, and nothing in any of them
 is ever synthesised here. **`telemetry-consent.json` and `setup-wizard-completed` gate
 registration**: a node missing either refuses to register and says which in its status
-document. `telemetry-contact.json` gates nothing and is optional throughout.
+document. `telemetry-contact.json` and `telemetry-claim.json` gate nothing and are
+optional throughout.
 
 **`telemetry-consent.json`** carries the three records `RegisterRequest.agreements`
 needs: `licence`, `remote_management` and `publication`. `publication` is a privacy
@@ -70,6 +71,13 @@ node that has none never calls the endpoint, which the spec states explicitly,
 so an absent file is a complete answer rather than a gap and nothing here
 blocks on it. See `collect/contact.py`.
 
+**`telemetry-claim.json`** carries the address that *owns* the node, for
+`PUT /nodes/claim`, plus a `send_requested_at` stamp when the owner asks for their
+link again. A different question from the contact email and a worse failure: the
+server mails this one a link, and opening it binds the node to the account behind
+it. Both keys optional, the file optional, and an unclaimed node runs exactly as a
+claimed one does. See `collect/claim.py`. Shipped in retina-gui 2026-09-22.
+
 **`setup-wizard-completed`** proves the config is the owner's rather than the shipped
 default. `retina-node/config/default.yml` ships a *working* configuration (Greenwich
 Observatory, Crystal Palace), the merger writes it on first boot, and retina-gui records
@@ -105,6 +113,7 @@ Nothing here is buildable from this repo, and the first one blocks every node:
 | Read `/data/retina-telemetry/status.json` | no, but | We bind no ports, so it is the only way *no identity*, *revoked token* and *rejected config* reach an operator. `telemetry_status.py` reads it and the home page shows it |
 | Collect `location.rx.beam_width` / `beam_azimuth` | no | Deferred indefinitely. Both are nullable, so sending two nulls is correct behaviour rather than a gap |
 | Collect the owner's contact details | shipped | Landed 2026-09-16. A skippable wizard step after the agreements step, plus a block under Remote support on the Configuration page, writing `/data/retina-gui/telemetry-contact.json` |
+| Collect the address that **owns** the node | shipped | Landed 2026-09-22. A Node claim section on the Configuration page writing `/data/retina-gui/telemetry-claim.json`, with Save for the address and Send again for another link. Not the contact email: see `docs/data-sources.md` §4 |
 
 `owl-os` separately owes a `mender-update show-provides` snapshot so
 `versions.retina_node` has a source. Optional field; omitted honestly until then.
@@ -141,6 +150,28 @@ Full detail and citations in `docs/data-sources.md`. The short version:
   Unlike a refused registration, it breaks nothing: the node registers, streams
   and beats exactly as before, and the only loss is a way to ring the owner.
   `detail` is for what stops a node working.
+- **The claim's address comes from retina-gui and nowhere else.** Never the contact
+  email: that answers "whom do we ring" and reusing it would mail a stranger a link that
+  hands them the node. `telemetry-claim.json` carries the address and, when the owner
+  presses send again, a timestamp. **Which call to make is decided here, not there**: a
+  changed address is a `PUT` and a fresh timestamp is a `resend`, because this is the
+  only side that knows what the server does with each.
+- **Offering an address the node already holds does nothing at all.** It is accepted,
+  writes nothing and mails nothing. A declined link leaves the node `unclaimed` *with
+  the address still on file*, so it can sit there and no number of offers will move it;
+  `POST /nodes/claim/resend` is the only way out. Checked against production on
+  2026-09-22. This is why the resend trigger exists, and why a "claim this node" button
+  that always PUTs is wrong.
+- **A stale ask for a link is ignored**, past `CLAIM_ASK_FRESH_FOR_S`. Nothing durable
+  records that we acted on one, so without an age bound every restart would mail the
+  owner another link.
+- **The three claim fields are read as one block, gated on `claim_state`.**
+  `claim_email` is required *and nullable*, so a field-by-field read would let a
+  detection ack, which carries none of them, blank an address the heartbeat reported a
+  second earlier. None of it gates anything: an unclaimed node registers, streams and
+  beats normally. **`pending` is not durable** and nothing should wait on it: a declined
+  link can strand a node there for about fifteen minutes before it falls back to
+  `unclaimed`.
 - **Nothing in the stack pushes to us.** No event bus, no inbound ports. Every input is
   a poll or a file read, including "the user changed the config".
 - **`wire/models.py` is generated.** Regenerate with `tools/generate-models.sh`; never
@@ -165,7 +196,7 @@ Full detail and citations in `docs/data-sources.md`. The short version:
   absence, so dropping the key produces a payload it rejects. `to_wire` also applies
   `mode="json"`, which is load-bearing: without it the acceptance timestamps stay as
   `datetime` objects and `json.dumps` refuses the registration payload outright.
-  **Fourteen fields are required-and-nullable in v1.2.2**, so payloads go out through
+  **Fourteen fields are required-and-nullable in v1.4.0**, so payloads go out through
   `wire.to_wire`, never `model_dump(exclude_none=True)` directly.
   `tests/wire/test_serialise.py` pins the inventory by name and fails if the spec grows
   or loses one.
@@ -233,8 +264,8 @@ that get re-litigated if the reasoning is not written down.
   beam fields were changed with the server author's agreement, relayed by Josh, and their
   next revision did not carry it, so our edit was silently reverted on adoption. **Check
   `NodeConfig.beam_width_deg` when adopting any revision**, and expect to reapply it.
-  Checked on adopting `1.2.2` (2026-09-16): it survived, nullable as agreed. Keep
-  checking anyway. Two revisions carrying it is not yet a habit.
+  Checked on adopting `1.2.2` (2026-09-16) and `1.4.0` (2026-09-22): it survived
+  both times, nullable as agreed. Keep checking anyway.
 - **The spec is the scope.** If a field is not in it, we do not collect it — however
   cheap or obviously useful it looks. Wanting something new means asking the server
   author, not a field we add unilaterally. This has already removed Pi

docs/data-sources.md

@@ -408,6 +408,63 @@ streams and beats exactly as before, and the only loss is a way to ring the owne
 Nothing is ever substituted, the same discipline as the consent records and the beam
 geometry. These reach a person.
 
+### The claim address
+
+`/data/retina-gui/telemetry-claim.json`, written by retina-gui's Node claim section and
+read-only to us. Two keys, both optional, and so is the file:
+
+| Key | Meaning | What it makes us do |
+|---|---|---|
+| `email` | the address that owns this node | a **change** is offered with `PUT /nodes/claim`, which is the call that mails a link |
+| `send_requested_at` | when the owner last pressed "send again" | a **fresh** one triggers `POST /nodes/claim/resend` |
+
+`email` is state and `send_requested_at` is an event, and keeping them apart is what
+lets retina-gui stay ignorant of the wire: it records what the owner wants, and
+`collect/claim.py` plus the config loop decide which of the two calls achieves it. The
+rule below is why that decision cannot live on the retina-gui side.
+
+**A stale ask is ignored**, past `CLAIM_ASK_FRESH_FOR_S` (five minutes). Nothing durable
+records that we acted on one, so without an age bound every restart of this container
+would re-read whatever the file last held and mail the owner another link, for ever.
+Somebody is looking at a page when they press that button, so an ask we were not running
+to see is one they will make again.
+
+**The contact email is not the claim address, however convenient that looks.** They are
+different questions with different consequences:
+
+| | `telemetry-contact.json` `email` | the claim address |
+|---|---|---|
+| Asks | whom to ring about this node | who *owns* this node |
+| If wrong | a support call goes astray | a stranger is mailed a link that hands them the node |
+| Optional | yes, entirely | there is no claim without one |
+
+An owner may well give the same address for both. That is their answer to two questions,
+not a licence for us to infer the second from the first, and reusing the contact email
+would claim ownership on behalf of whoever happened to be listed. The same discipline as
+the consent records: nothing that reaches a person is ever synthesised here. The two
+live in separate files and neither is ever read as a fallback for the other.
+
+**Offering an address the node already holds does nothing at all.** Not "nothing
+visible": the server accepts it, writes nothing and mails nothing, and the state it
+reports is unchanged. That matters because of what a declined link leaves behind,
+verified against production on 2026-09-22: declining returns the node to `unclaimed`
+*but leaves the address on file*, so a node can sit unclaimed with an address against it
+and no number of offers will ever move it. `POST /nodes/claim/resend` is the only way
+out. This is the single most surprising thing about the claim and the reason the resend
+trigger exists at all.
+
+**What a node *is* told, since v1.4.0**, is where its claim stands: `claim_state`,
+`claim_email` and `claim_undeliverable`, restated on every heartbeat and contact
+response. Those three are read (`comms/levels.py`) and written to the status document,
+which is the only way they reach an owner. None of them gates anything: an unclaimed
+node registers, streams and beats exactly as an owned one does.
+
+`claim_state` is `unclaimed`, `pending` or `owned`. **`pending` is not durable.** A
+claim link declined while a second is outstanding can leave a node reading `pending`
+against a link nobody can redeem, until the challenge expires about fifteen minutes
+later and it reads `unclaimed` again. That is a known server-side race, tracked there.
+Nothing here should wait on `pending` or treat reaching it as progress.
+
 ### The agreements, and the publication choice
 
 `RegisterRequest.agreements` requires three records. Today: