Skip to content

Coverage wedge: where the node is proven to see, from ADS-B-verified detections - #38

Merged
Purple10101 merged 4 commits into
mainfrom
20261017-coverage-wedge
Oct 19, 2026
Merged

Purple10101 merged 4 commits into
mainfrom
20261017-coverage-wedge

Conversation

@Purple10101

@Purple10101 Purple10101 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a coverage wedge: the region around the receiver where the node is proven to detect aircraft, built only from ADS-B-verified detections, never from a declared antenna aim. Alongside it, each unlabelled track's delay ellipse is cut against the wedge (solid inside, faint outside). retina-gui's Tracker page draws both on a map (companion PR in retina-gui).

Tracking is unchanged: this adds a recorder beside the tracker, positions on the history stream, and new endpoints.

What changed

  1. Verified positions only. A detection counts when blah2 matched it to the track's own aircraft (the detection's tag, since a track keeps its hex after a swap) and projected the fix to the detection's time, at most 20 s, blah2's own projection cap. Never the tracker's estimate.
  2. The wedge. Passes are binned by bearing in 5° slices. A pass needs 3 detections, and each slice holds the second-furthest range two different aircraft reached over a sliding 24 h, so one wrong ADS-B match cannot push it out. Slices are drawn edge to edge.
  3. Processed range limit. Where a slice reaches the ellipse of blah2's delayMax, its reach is the node's configuration, not the antenna. /coverage serves that ellipse (span_ring) and flags such slices (limited).
  4. Close-in slices (held under 10 km) are served apart (close_lobes): close to the receiver they are mostly aircraft overhead, which a flat map cannot tell from the horizontal pattern.
  5. Arcs, flat as retina-server's are, cached by differential; a replay gets at most 240 spread over a track.
  6. Persistence. Saved to coverage.json beside the -s events file every 60 s and on reset; restarts keep the wedge (unlike events.jsonl). It starts again when the receiver or transmitter moves.
  7. Endpoints: GET /coverage, GET /coverage/arcs, GET /coverage/track/<id>, POST /coverage/reset.

This deliberately does not port retina-server's evidence-only polygon: its point-count thresholds measure dwell time rather than independent passes, its 200-point FIFO cannot express a time window, and centre-of-bin vertices draw every lobe 5° narrower than its evidence.

Measured on jonathan-node-1

Live since 17 October (latest commit since 19 October 10:51 UTC).

Day 1 Day 2
Slices proven 39 54 (36 close-in)
Aircraft proving them 127 273
Main lobe 115-170° 100-185°
Slices at the processed range limit 7 (130-165°) 7 (135-165°)
  • Fix age: at 10 s the rule refused 22% of correctly tagged detections, since ADS-B arrives in batches about 8.6 s apart. Measured as the jump when a fresh fix replaces a projected one, blah2's projection error was a median 0.07-0.14 km (90th percentile about 0.2 km) at every age up to 20 s. Raising the limit to 20 s recovered 25% of tagged detections.
  • Range: the main lobe's held ranges sit at 87-98% of the delayMax ellipse, so jn1's reach there is set by its delay span.
  • Close-in: outside the lobe, detections within 8 km sat at a median 45° elevation.
  • Cost: about 1% CPU; /coverage 6-24 ms; latest arcs 2-5 ms; a replay of the longest track 108 KB in 39 ms; coverage.json 115 KB at a full day.

Compatibility

  • History stream: track point-sets gain lat/lon (null where unverified). Detection classes are unchanged; a consumer that ignores unknown keys is unaffected. Track points cost 28 bytes instead of 20 in /health's estimate.
  • Config: a new coverage section, all defaulted (bin_deg, hold_s, max_position_age_s, min_pass_detections, close_in_km). location is read from the blah2 config the tracker already loads. New --coverage-state flag, defaulting beside the events file.
  • No retina-node change is needed beyond the image tag.

Known limitations and follow-ups

  • The constants (min_pass_detections 3, max_position_age_s 20, close_in_km 10, the 0.85 limit fraction) were set and validated on jonathan-node-1 only; fairforest B is next. jonathan-node-2 has no ADS-B labels, so it cannot build a wedge yet.
  • Arcs are flat, so an aircraft at altitude sits a few km off its arc. A 3D wedge is the planned part two.
  • Without the full ADS-B aircraft list a slice cannot tell "no traffic" from "traffic not detected"; deferred.
  • GET /frame logs a BrokenPipe traceback when a client hangs up mid-reply; pre-existing, not from this change.

Docs

docs/coverage-wedge.md (new), CLAUDE.md's test list, and the storage note in history.py.

Verification

ruff check (E,F,W and the repo config), ruff format --check and pytest (417 passed) clean; the dead-code gate is clean. Checked end to end against a synthetic feed over TCP, and live on jonathan-node-1.

After this

🤖 Generated with Claude Code

Purple10101 and others added 3 commits October 17, 2026 13:59
…ions

A node had no way to show where it can actually see. The only coverage
picture available was a declared beam, which is configuration, not
measurement. The tracker now records where ADS-B-verified detections land
and serves a wedge of proven coverage, with each unlabelled track's delay
ellipse cut against it, for retina-gui's Tracker page to draw.

A position counts only when blah2 matched the detection to the track's own
aircraft and projected the fix to the detection's time, at most 10 s
stale. Passes are binned by bearing in 5 degree slices, and each slice
holds the second-furthest range two different aircraft reached over a
sliding 24 hours, so one wrong ADS-B match cannot push the wedge out. This
deliberately does not port retina-server's evidence-only polygon, whose
point-count thresholds measure dwell time rather than independent passes.

The receiver and transmitter come from blah2's own location block, which
the tracker already reads. The wedge is saved beside events.jsonl so a
restart keeps it, and starts again when either end moves or on POST
/coverage/reset. Track points on /events now carry their verified lat/lon.
Arcs are flat, as retina-server's are. New endpoints are GET /coverage,
/coverage/arcs and /coverage/track/<id>.

Covered by tests/test_coverage.py (46 tests) and an updated history stats
test; 410 pass. Design in docs/coverage-wedge.md. Deployed by hand to a
test node for live trials; the thresholds are not yet validated across
sites.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… keep good fixes

On a test node's first day the main lobe's held ranges sat at 87-98% of
the furthest bistatic range blah2 processes (its delayMax), so the wedge
was reporting the node's configuration there and reading as antenna
reach. /coverage now serves that limit as span_ring, the ellipse of the
furthest processed differential, and flags each slice whose held range
reaches 85% of it (limited, limited_bins), so the page can say which
slices are capped by the setting rather than by the antenna.

The 10 s fix-age limit came from retina-server, where it bounds an
unprojected fix. blah2 projects every fix to the detection's time, and
on that node the limit refused 22% of correctly tagged detections, since
ADS-B arrives in batches about 8.6 s apart and ages run up to 20 s.
Measured on the same recording, as the jump when a fresh fix replaces a
projected one, the projection error was a median 0.07-0.14 km at every
age up to 20 s, far inside a 5 degree slice. The limit is now blah2's own
20 s projection cap.

Six new tests cover the limit, the ring and the flags; 416 pass.
docs/coverage-wedge.md records the measurement and the new section.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a test node's second day the wedge proved small slices (1-8 km) in
almost every direction around the receiver, including straight away from
the main lobe. Detections there sat at a median 45 degrees elevation:
aircraft passing nearly overhead, which a flat map cannot tell from the
antenna's horizontal pattern. Filtering by elevation does not separate
them cleanly, because on a short-reach node with traffic 4-5 km up the
main lobe's own detections sit at 26-30 degrees.

So slices are split by ground range instead. A slice held under
coverage.close_in_km (10 km) stays proven and counted, but /coverage
serves it in close_lobes rather than lobes and flags it in close_in, so
the page can draw it as close-in overhead traffic. 10 km clears the
furthest ring slice seen (8.1 km); 8 km left it as a solid spike. It was
set from one site and should be checked on others. Handling elevation
properly is the 3D wedge.

A new test covers the split; 417 pass. docs/coverage-wedge.md gains a
Close-in slices section.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Claude finished @Purple10101's task in 34s —— View job


Review of PR #38: coverage wedge

I read the diff by static review only. I did not run ruff or pytest, and the code-review skill failed to launch, so this is a manual read of coverage.py, control.py, history.py, server.py and config.py. I did not read tests/test_coverage.py or docs/coverage-wedge.md.

The design is sound. Only ADS-B-verified positions feed the wedge, and the tracker is untouched. The sliding second-best-range rule, the lock discipline and the input validation (verified_position, Geometry.from_location, the TRACK_ID regex) all look correct. I found no blocking bugs, only the points below.

Worth fixing

  1. Save race on the shared temp file (coverage.py:453-462). save() is called from the saver thread every 60 s and from reset() on an HTTP thread. Both write to the same state_path + ".tmp". Two overlapping saves can interleave writes into one file, or the second os.replace can fail with FileNotFoundError (an OSError, so it is only logged). A dedicated save lock, or a per-call temp name via tempfile.mkstemp(dir=...), fixes it.
  2. No final save on shutdown. A restart loses up to 60 s of passes. The PR says as much, so this is acceptable if intended. A save from a signal or exit handler would make restarts lossless.
  3. load() doesn't cover every malformed file (coverage.py:470-473).
    • If the JSON parses but isn't an object, saved.get raises AttributeError, which the except (OSError, ValueError, TypeError) doesn't catch. Startup would crash on a corrupt coverage.json.
    • json.load accepts NaN/Infinity, so Pass.from_list could accept a non-finite range_km and held would then yield a non-finite range. Reject non-finite values in from_list.
    • Add AttributeError to the except, or check isinstance(saved, dict).

Minor

  1. Dedupe high-water mark can be evicted (coverage.py:226-227). _through holds at most 512 tracks. If a track is evicted while events still repeat its window, the repeated detections are counted again, which inflates n and could pass the 3-detection threshold. This is unlikely, but a distinct-timestamp set per pass would remove the edge case.
  2. Out-of-order timestamps are dropped silently. ts <= through skips late detections that were never counted. That matches the history's behaviour, so it is consistent, but it is worth a line in the docs.
  3. Arc cache churn. The cache key includes the whole held tuple, which changes at most every 10 s (HELD_REFRESH_MS). Each change invalidates every arc. The 4096 cap bounds memory, but a busy /coverage/arcs poll recomputes everything after each change. Keying on a hash of held would be no better, so this is only a note on the cost you measured.
  4. Style. CLAUDE.md and .claude/rules/code-style.md say "no comments". The new code adds many docstrings and # ── Section ── banners (coverage.py, control.py, history.py). The surrounding files already carry comments, so this is a judgement call, but the banners are the easiest to drop.
  5. Unauthenticated reset. POST /coverage/reset wipes up to 24 h of evidence. The control server defaults to loopback, so this is fine as long as that default holds. The docs should state that the endpoint is as trusted as /reset.

Verified OK

  • ellipse_range_km has a strictly positive denominator, so there is no divide-by-zero at any bearing.
  • _build_arc's bin_deg / 2 step matches the (k // 2) bin lookup.
  • TeeEventWriter drops None writers, so coverage-disabled runs are safe.
  • DELAY_MAX_KM exists in config.py.

…e file

Review of #38 found two real faults in how the wedge is saved and
restored. A coverage.json that parsed but was not an object, or held a
non-numeric start time, raised out of load(), which runs at startup; under
the node's restart: always that is a crash loop until someone deletes the
file. Reproduced on a test node with the real entry point: the tracker
exited 1 with an AttributeError at load. Damaged values elsewhere (a
non-numeric reset time, an infinite range) were trusted in part.

The state file is now validated whole before anything in it is used, and
any failure leaves the wedge to start fresh. On the same node, with the
same damaged file, the fixed tracker logs that it is ignoring it, serves
/coverage and saves a valid file on reset.

Saves also raced. The saver thread and a reset both wrote the same temp
file, so a save that took its snapshot before a reset could land after
it, leaving the pre-reset wedge on disk for the next restart to bring
back. Saves are now serialised and snapshot inside the lock.

The section banners in coverage.py are dropped, per the no-comments rule,
and the docs record the dedupe bound, late detections and why the reset
endpoint is trusted. 14 new tests reproduce both faults (8 failed before
the fix); 431 pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Claude finished @Purple10101's task in 19s —— View job


Review of PR #38: coverage wedge

The code-review skill failed to launch again, so this is a manual read of retina_tracker/coverage.py at the current HEAD. I did not run ruff or pytest, and I did not re-read the other files.

No blocking issues found.

Earlier review items that HEAD already fixes

  • Save race. save() now takes _save_lock (coverage.py:453), so overlapping saves from the saver thread and reset() can't collide on the temp file.
  • Malformed coverage.json. _read_state (coverage.py:487-501) checks that the file is an object, that passes is a list, that the timestamps are finite, and that reset_reason is a string. Pass.from_list rejects non-finite values (coverage.py:153-157). A damaged file is ignored whole rather than half-trusted.

Still open (minor)

  1. Dedupe high-water mark can be evicted (coverage.py:224-228). _through holds at most HIGH_WATER_MEMORY tracks. If a track is evicted while events still repeat its window, those detections are counted again. That inflates n and could carry a pass over min_pass_detections. It is unlikely, but a short note in the docs or a larger cap would cover it.
  2. No save on shutdown. A restart loses up to 60 s of passes. The saver is a daemon thread, and a final save() from an exit handler would make restarts lossless. This is acceptable if intended.
  3. Out-of-order detections are skipped. ts <= through (coverage.py:218) drops late detections that were never counted. That matches the history's behaviour, but the docs should say so.
  4. Style. CLAUDE.md and .claude/rules/code-style.md say "no comments". The new code adds many docstrings in coverage.py. The surrounding files already carry some, so this is a judgement call.
  5. Reset endpoint is unauthenticated. POST /coverage/reset wipes up to 24 h of evidence. That is fine while the control server stays on loopback, but the docs should say the endpoint is as trusted as /reset.

Checked and OK

  • _build_arc: the bin_deg / 2 step matches the (k // 2) bin lookup, and the crossing interpolation g0 / (g0 - g1) only runs when the signs differ, so there is no divide-by-zero.
  • _held_bins takes the second-best range per bin across distinct aircraft, as described.
  • verified_position requires the detection's tag to match the track's hex, a finite position_age within the limit, and finite, in-range coordinates.
  • Locking: arc() releases the lock while building and re-acquires it to store, which is safe because the build is pure.

@Purple10101

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Fixed in ee13192, point by point:

1. Save race: fixed. It was worse than a logged error: a save that snapshotted before a reset could land after it, leaving the pre-reset wedge on disk for the next restart to bring back. Saves are now serialised by a save lock, and each takes its snapshot inside it. test_a_reset_during_a_save_is_what_ends_up_on_disk forces that interleaving: before the fix the file ended as the pre-reset wedge (reset_reason None); after it, it ends as the reset left it. It passed 25 of 25 repeated runs.

3. Damaged state file: fixed. This was the serious one. load() runs at startup, so a file that parsed but was not an object (or held a non-numeric start time) crashed the tracker, and under restart: always that is a crash loop. The file is now validated whole (_read_state), and any failure starts the wedge fresh rather than trusting it in part. Non-finite or mistyped pass values are refused in Pass.from_list.

  • Tests: 12 damaged-file cases plus a good-file control. Before the fix, 4 raised out of load() (AttributeError x3, ValueError x1) and 3 were partly trusted. All pass now.
  • Live on jonathan-node-1, using the real entry point in a throwaway container with the node's image, config and a [1, 2, 3] coverage.json:
    • before: AttributeError: 'list' object has no attribute 'get' at coverage.py:472, exit status 1;
    • after: Ignoring unreadable coverage state ...: not a JSON object, the tracker runs, /coverage answers, and POST /coverage/reset writes a valid file.
  • Redeployed to jonathan-node-1 afterwards. The real wedge loaded unchanged (54 slices, 278 aircraft) with no warning, and the next timed save wrote a valid file (2,321 passes, no leftover .tmp).

7. Style: the section banners in coverage.py are gone. The ones in history.py and control.py predate this PR, so I left them.

5 and 8: now documented in docs/coverage-wedge.md. Late detections are dropped as the history drops them. POST /coverage/reset is trusted on the same footing as POST /reset: the control surface binds to loopback.

Kept as is, with reasons in the docs:

  • 2. No save on shutdown. It loses at most a minute of a 24 h hold, which is not worth changing how the tracker stops.
  • 4. Dedupe eviction. It uses the same 512-track bound as TrackEventWriter, and needs more than 512 tracks emitting at once to bite.
  • 6. Arc cache. The cache key is the held ranges' values, not a timer, so recomputing the wedge every 10 s only misses the cache when a slice actually changes. On jonathan-node-1, latest arcs take 2-5 ms.

431 tests pass; ruff, format and the dead-code gate are clean.

🤖 Generated with Claude Code

@Purple10101
Purple10101 merged commit e0870bc into main Oct 19, 2026
2 checks passed
@Purple10101
Purple10101 deleted the 20261017-coverage-wedge branch October 19, 2026 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant