Skip to content

feat(battery): repeater battery readings from observers that poll their neighbours - #2265

Open
A13xB0 wants to merge 21 commits into
Kpa-clawbot:masterfrom
A13xB0:feat/neighbour-battery
Open

A13xB0 wants to merge 21 commits into
Kpa-clawbot:masterfrom
A13xB0:feat/neighbour-battery

Conversation

@A13xB0

@A13xB0 A13xB0 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Problem

Repeater adverts don't carry battery, so CoreScope only knows a repeater's battery when the repeater is itself an observer. Observers can ask the repeaters they hear directly: a guest login (blank password works on default repeaters) plus REQ_TYPE_GET_STATUS returns batt_milli_volts. There was nowhere to send those readings.

Change

  • Topic: meshcore/<region>/<observer>/battery, one batch per poll cycle:
    {"v":1,"origin":"…","origin_id":"<observer key>","timestamp":"…",
     "readings":[{"node_id":"<64-hex>","battery_mv":3940,"snr":-5.0,"rssi":-119,
                  "source":"poll","reply_hops":0,"method":"status","sampled_at":"…"}]}
  • Ingestor: validates each batch and stores readings in a new neighbor_battery table (one row per node, observer and sample). Rejects: unknown version, origin_id ≠ topic, 0 or > 64 readings. Skips a single reading for: bad key, self-report, mV outside 0–10000 or not an integer, sampled_at more than 2 h old or 5 min ahead, duplicate node, bad source/method/hops. At most one stored reading per node per observer every minIntervalMinutes (default 10). Handled next to /neighbors, after the blacklist and IATA whitelist.
  • API: /api/nodes/{pk}/battery merges the node's own status reports (source: self) with neighbour readings (source: neighbor, with observer_id, observer_name, snr, reply_hops). New sources (per source: latest, count, median SNR) and selected (the source latest_mv and status come from: a self report under 2 h old, else the neighbour observer with the best median SNR over 24 h, else the newest). status gains no_battery for 0 mV, which is what an openHop repeater without a power sensor reports; it was graded critical before.
  • UI: the node analytics battery card names the observer behind the value (linked, with route, SNR and age) and lists any others, draws one line per source on a time axis, and lists recent readings with observer, SNR and route.
  • The route is now in the OpenAPI spec (removed from the known gaps).

Config

Opt-in, ingestor side:

"neighborBattery": { "enabled": true, "minIntervalMinutes": 10 },
"retention": { "neighborBatteryDays": 30 }

Off, the topic is dropped and nothing changes. The server works with or without the table.

Scale: a reading is one row per node per observer per poll (hourly by default). Reads are by node_id, the primary key prefix.

Tests

  • Ingestor: storing a reading, keeping 0 mV, every message-level reject, every reading-level skip, duplicates, defaults, the min-interval cap per observer, prune, config defaults, and the topic gate through handleMessage (off stores nothing; on stores and doesn't touch observers).
  • Server: neighbour samples with observer names, best-SNR selection over the last reporter, a fresh self report winning and a stale one not, no_battery, a missing table, and the fallback to the newest source.
  • Frontend: test-node-analytics-battery-observers.js (Reported by, one line per observer, readings table, escaping, self-only).
  • go test ./... passes in cmd/ingestor and cmd/server; test-all.sh passes.

A publisher for openHop repeaters is being built alongside this; until one exists, nothing publishes on the topic.

@efiten efiten left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I read the whole diff and ran go vet, gofmt -l and the cmd/ingestor and cmd/server suites on 280a3c79 (all pass), plus test-all.sh (passes). The firmware side matches: batt_milli_volts is the first uint16_t of RepeaterStats (examples/simple_repeater/MyMesh.h:44-45), in mV, and 0 mV means no sensor on boards without one (STM32Board.h:21), so no_battery is right. The writes stay in the ingestor and the server only reads.

Three things to change before merge:

  1. The observer ranking uses a different window than documented. The PR body and openapi.go:150 say "best median SNR over 24 h". summariseBatterySources (cmd/server/neighbor_battery.go:136-163) takes the median over the whole ?days window (up to 365); the 24 h check in selectBatterySource (:194) only looks at each observer's latest reading. So selected depends on days. A probe test (not committed) showed it: observer A with 20 readings 3-4 days ago at SNR +10 and one reading 1 h ago at -15, observer B with one reading 2 h ago at +5. A is selected (median 10), and A's latest reading sets the status. Computing the median only over readings inside batteryRankWindow would match the docs.
  2. Any publisher can set any node's battery. Readings are accepted from whoever may publish on meshcore/+/+/battery, for any node_id. The origin_id check at cmd/ingestor/neighbor_battery.go:64-68 compares the payload with itself; the comment says the broker binds origin_id to the authenticated key, but nothing in the repo enforces or documents that. SNR is accepted up to +64 (:151) and the highest median wins, so one publisher reporting snr: 64 overrides every real neighbour. Could you add a doc page with the broker ACL this needs (as docs/client-rf-samples.md:65 does for the client topics), bound SNR to a realistic range, and only accept readings from observers present in observers?
  3. Retention defaults to keep forever. config.go:381-389 falls back to 0 when the feature is on and neighborBatteryDays is absent; only config.example.json sets 30. minIntervalMinutes limits per (node, observer) pair, so a publisher rotating observer keys adds up to 64 rows per message. A non-zero default (30) when the feature is enabled would bound it.

Smaller, can be in the same push:

  1. public/node-analytics.js:366 adds BATTERY_LINE_COLORS as hex values, and node-battery.js:11-14 puts hex fallbacks in var(). Colours go through CSS variables in this codebase.
  2. node-battery.js:17-26 (route, reporter) repeats batteryRoute and batteryObserverLink from node-analytics.js:380-390.
  3. fillObserverNames (cmd/server/neighbor_battery.go:102) runs one query per observer; one IN (...) query does the same.
  4. The PR adds 4 map[string]interface{} uses outside tests (parseBatteryReport, parseBatteryReading, handleBatteryReport, one cast in neighbor_battery.go). AGENTS.md asks for that count to go down; a named struct for the payload would do it.

Not done on my side: no browser check, and nothing publishes on this topic yet, so I tested only through the unit tests and the probe above.

A13xB0 added 19 commits October 25, 2026 22:53
An observer that polls the repeaters it hears directly publishes their
battery on meshcore/<region>/<observer>/battery. The ingestor validates
each batch (version 1, origin_id matches the topic, 1-64 readings, 64-hex
node keys, no self-reports, 0-10000 mV, sampled_at within 2 h before to
5 min after receipt) and stores readings in neighbor_battery, one row per
node, observer and sample, at most one per node per observer every
minIntervalMinutes. 0 mV is kept: it means no battery fitted.

Opt-in with neighborBattery.enabled; retention.neighborBatteryDays bounds
the table.
/api/nodes/{pk}/battery merges the node's own status reports with the
readings observers published for it. Each sample says its source and, for
a neighbour reading, the observer, its SNR and reply hops. sources
summarises each one and selected names the one latest_mv and status come
from: a self report under 2 h old, else the neighbour observer heard best
over 24 h, else the newest. 0 mV is now status no_battery.

Without neighbour readings the response is unchanged apart from the new
fields. The route is now in the OpenAPI spec.
A Reported by line names and links the observer behind the latest value,
with route, SNR and age, and lists any others. The chart draws one line
per source on a time axis, and a table lists the recent readings with
their observer, SNR and route.
…drawn

The status badge set its icon markup as text, so the card showed the raw
<svg> tag. A single reading gave the chart's time axis no width and the
point was not drawn; the axis now spans an hour either side.
The latest voltage, its status, who reported it (the node itself or the
observer that polled it, linked), the route and its age, with a link to
the history on the Analytics page. Hidden when the node has no readings.

The analytics battery card keeps its readings and thresholds off the
chart's edges, and its badge icon is no longer stretched by the card's
chart SVG rule.
…attery readings

A /battery reading may now carry temperature_c (the repeater's chip
temperature from a telemetry request), via (the relay a one-hop request
went through) and stats (the rest of the MeshCore status reply). The first
two go in new neighbor_battery columns, added to existing tables by
migrateNeighborBatteryV2; stats go in a new neighbor_status table beside
the battery row and are pruned with it. Each optional field is checked on
its own, so an odd value is dropped without losing the reading.
/api/nodes/{pk}/battery adds temperature (an observer's chip-temperature
reading, else the node's advert temperature), via/via_name on one-hop
readings, and stale flags for readings older than
neighborBattery.hideAfterHours (default 24, 0 never). The data is still
returned; the UI hides stale lines.

New /api/nodes/{pk}/status-history returns the status replies observers
published, one series per observer, with hourly rates, the flood
duplicate rate and reboots worked out. Both read older tables without
the new columns.
The node page and side panel show a Temperature line under Battery,
labelled as the chip temperature and naming who reported it. Either line
hides on its own once its reading is older than hideAfterHours, and
neither appears for a node with nothing recorded.

On the analytics page the battery chart gains a chip temperature trace and
the readings table a column for it; one-hop readings name their relay. A
Repeater Status card charts uptime (reboots marked), noise floor, last
RSSI/SNR, airtime, packets received and sent, flood duplicate rate,
receive errors and TX queue. It appears only when an observer has
published status stats.
master has its own getJSON test helper; the two clash once merged.
With one status reply Chart.js picked a span of months; pin the axis to
the samples, padded by an hour either side for a lone one, as the battery
chart does.
status-history marks a series stale once its newest sample is older than
neighborBattery.hideAfterHours, and the analytics card draws the best
fresh series or stays hidden. A repeater an observer now reads by
telemetry only (one hop away) stops sending status, so its charts would
otherwise sit there frozen.
A reading an observer took under the shared guest key carries "key":"shared".
Store it as neighbor_battery.key_mode, carry it through the battery API, and
show "shared key" beside the observer on the node page. Readings under an
observer's own key are unchanged (key_mode NULL).

- ingestor: parse the key field, key_mode column, migrateNeighborBatteryV2
  adds it alongside temperature_c and via.
- server: key_mode on NodeBatterySample and BatterySource.
- frontend: "shared key" in the Battery line credit.
Commit the one public shared guest key as a constant, the same value the
Neighbour Battery plugin compiles in, and serve it at /api/shared-guest/key so
a client can poll under it without pasting. The key is public by design, so the
whole keypair is in the clear; the Phase 2 decoder will decrypt replies with it.
A repeater reply under the shared guest key is encrypted to the shared key,
which CoreScope holds, so the ingestor can decode it out of any observer's
uploaded raw packet — that observer runs nothing special. For a RESPONSE
addressed to the shared key's 1-byte hash (~1/256 of packets), try each known
repeater with the source hash, verify the 2-byte MAC, decrypt, and store the
battery/telemetry reading credited to the observer that heard it (key_mode
shared).

Crypto is standard library only: RFC 7748 X25519 over math/big plus the
Ed25519->Montgomery map, cross-checked byte-for-byte against openhop_core.
Reply kind (status vs telemetry) is told by shape; a login reply fits neither
and is ignored.
A shared-key reply decoded from N observers was stored N times — same repeater,
same value — so the battery chart drew N identical lines. The reply's tag is the
same in every observer's copy, so key the reading by it (sampled_at = the tag's
time) and credit it to the shared key itself rather than each observer. All
copies of one reply now collapse to a single "shared key" reading. The node page
shows "via the shared key".
A shared-key reply is decoded from whichever observers overheard it, but it was
requested by one poller. The poller claims the nodes it polls under the shared
key (meshcore/<region>/<observer>/sharedclaim); the ingestor remembers the
claim (3 h TTL) and credits a decoded reply to the claimant rather than the
incidental observer or the shared key. The node page then reads "requested by
<poller>", falling back to "via the shared key" when no claim is known.
- Claims carry the poller's route (hops/via); a decoded reply uses it, so a
  reading is no longer shown as "direct" when it isn't. Unclaimed replies store
  no route and the UI leaves it out.
- Reject a reply tag outside the /battery window (2 h past, 5 min future), so a
  bad clock or a wrong candidate can't write a far-future "latest" row; stamp
  received_at at the real arrival time, not the tag.
- HandleClaim checks origin_id against the topic (no spoofing), evicts lapsed
  claims and bounds the map.
- Cache each repeater's ECDH secret (the seed is fixed) and run the candidate
  refresh off the lock, so the hot path stays cheap.
- Telemetry needs a real voltage (reject 0 mV); add the missing CayenneLPP
  type-1 size.
- Frontend: node-battery, node-analytics and node-status-charts show
  "requested by <poller>" or "via the shared key", never a link to a
  non-existent observer and never "direct" for an unclaimed shared reading.
@A13xB0
A13xB0 force-pushed the feat/neighbour-battery branch from 4ae22fc to 6d848e9 Compare October 25, 2026 22:54
Record every observer whose packet CoreScope decodes under the shared
key (neighbor_battery_heard), and list them as 'heard by ...' on the
battery reported-by line, alongside the poller credited with the
reading. Also document /api/shared-guest/key in the openapi spec.
@A13xB0
A13xB0 force-pushed the feat/neighbour-battery branch from 5222376 to 92831a1 Compare October 26, 2026 01:32
@n30nex

n30nex commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Rechecked the existing review against 92831a19. The ranking-window issue still reproduces in the current server code: the same recent readings select observer B at 4000 mV for days=1, but observer A at 2900 mV for days=7 because old +10 dB readings enter the median. The 24-hour filter is still applied only to each source's latest timestamp. Please exclude older readings from the ranking median and retain them only for chart history. The other six review requests also remain visible in this head; I would keep the existing review open.

@efiten efiten left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the new commits. I re-read the diff at 92831a19 and ran go test -run 'Shared|Battery|Claim|Status' in cmd/ingestor and -run 'Battery|SharedGuest|StatusHistory' in cmd/server (both pass). The RepeaterStats field order in sgStatusFields matches examples/simple_repeater/MyMesh.h:44-60, and the RESPONSE layout matches docs/payloads.md:69-78.

The earlier review

None of the seven points has changed yet:

  1. Ranking window: summariseBatterySources (cmd/server/neighbor_battery.go:238-267) still takes the median over the whole ?days window, and the 24 h check in selectBatterySource (:299) still looks only at each source's latest reading.
  2. Publisher trust: SNR is still accepted up to ±64 (cmd/ingestor/neighbor_battery.go:196), there is no check that the observer exists in observers, the origin_id check (:110-114) still compares the payload with its own topic, and there is no doc page with the broker ACL.
  3. Retention: NeighborBatteryDaysOrZero (cmd/ingestor/config.go:384-389) still returns 0 when the setting is absent.
  4. Hex colours: still there, and more were added (public/node-analytics.js:373, :488, :529-533, node-battery.js:13-15, node-status-charts.js:41-43, :104).
  5. route / observerLink in node-battery.js:19-35 still repeat node-analytics.js:393-405.
  6. fillObserverNames (cmd/server/neighbor_battery.go:142-157) still runs one query per observer, and the new fillNodeNames (:130-137) does the same per relay.
  7. map[string]interface{} outside tests went from 4 to 14 added lines, including HandleClaim (shared_guest_decode.go:121, :140) and the response in shared_guest_key.go:22.

The shared guest key

  1. Question on the trust model. The keypair is committed in cmd/ingestor/shared_guest_decode.go:28-29 and cmd/server/shared_guest_key.go:15-16, and GET /api/shared-guest/key serves the private half with no auth and no config gate. The comment says this is public by design. The consequence: anyone with that seed can derive the ECDH secret with any repeater, build a valid RESPONSE "from" it and inject it on any packets topic or over RF. Handle (shared_guest_decode.go:232-295) stores it as a reading if the 2-byte MAC, the tag window and the mV value check out. The same seed also gives everyone the access level of every repeater that adds this identity to its ACL. Is that acceptable for battery readings that drive the node status? If yes, it needs to be written down for operators, and the readings should be marked as unauthenticated. If no, a per-deployment key from config, not served by the API, would close it.
  2. The decoder runs with the feature off. main.go:106 creates it unconditionally, and :1040 (HandleClaim) and :1300 (Handle) call it without cfg.NeighborBatteryEnabled(), unlike the /battery topic at :1030. With the default config, rows land in neighbor_battery and neighbor_battery_heard while pruning never runs (retention 0).
  3. Claims can be taken over. Any client that may publish on meshcore/<r>/<X>/polling can claim any node_id. The last writer wins (shared_guest_decode.go:164), and the decoded reading is then credited to X, which does not have to be a known observer.
  4. Stored XSS through the relay name. batteryRoute (public/node-analytics.js:396) inserts s.via_name unescaped, and :446 writes the result with innerHTML. via_name is the relay's advert name, which sanitizeName (cmd/ingestor/decoder.go:1195) does not HTML-escape, and via is chosen by the publisher of the claim or the /battery payload. node-battery.js:26 already escapes the same field, so escapeHtml here fixes it.
  5. CI is red because of gofmt. "Go Build & Test" fails on cmd/ingestor/shared_guest_decode.go (the struct at :44-48 and the const block at :71-76), and "Build & Publish Docker Image" fails because it depends on that job. The E2E shards passed.

Smaller:

  • sgTagTTL (:39) is unused.
  • The key constants exist twice, in the ingestor and the server.
  • sgParseStatus stores the stats without the range checks statusFields applies.

Not done on my side: I did not build a forged packet end to end (points 1 and 3 come from reading the code), and no browser check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants