Skip to content

bugfix: Mic-E - stop rejecting valid Mic-E longitude/speed bytes - #568

Merged
kotfu merged 5 commits into
chrissnell:mainfrom
zwolanj:bugfix/mic-e-decoding-fix
Sep 24, 2026
Merged

kotfu merged 5 commits into
chrissnell:mainfrom
zwolanj:bugfix/mic-e-decoding-fix

Conversation

@zwolanj

@zwolanj zwolanj commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Two independent false-positive rejections were dropping/flagging valid Mic-E packets, discovered while diagnosing why NX0R-7's beacons were never gated to APRS-IS despite decoding fine on other iGates.

pkg/aprs/mice.go (decodeMicELon):
decodeMicELon rejected 0x7f (DEL) unconditionally in any of the 3
Mic-E longitude bytes as an "ambiguous/no GPS fix" sentinel. That
check was added (issue #76) to guard against a specific wraparound
danger (DL9DAK/DL8XI reports) that only occurs when DEL combines
with the destination's +100 degree longitude-offset bit. When the
offset bit is NOT set, 0x7f is just the ordinary top-of-range
digit-99 encoding (byte = digit + 28) and decodes to a real,
plausible position. NX0R-7's on-air packets have offset=0, so this
unconditional check was silently dropping every one of its beacons
before they ever reached the RF->IS gate.

Fix: keep 0x20 (SPACE) always rejected in the longitude field -
it's the APRS101 ch 10 spec-reserved "unknown data" sentinel and
has no offset-dependent nuance. Reject 0x7f (DEL) only when the
destination's longitude offset is actually 100, matching the exact
real-world bug pattern the check was written for.

Added TestParseMicEDelInLonAcceptedNoOffset using the real
NX0R-7 packet bytes (dest S8UR7Y and S8UV2X, offset=0) asserting
they now decode to sane coordinates instead of erroring. Existing
TestParseMicEDelInLonRejected (DL9DAK/DL8XI, offset=100) and
TestParseMicEAmbiguousLonRejected (SPACE, offset=100) continue to
pass unchanged.

web/src/lib/packetInspect.js (validateMicE):
The web packet inspector's Mic-E byte-range sanity check used
0x26-0x7F as the "encodable range", which is wrong - the real
encoding (byte = digit + 28) legally produces bytes as low as
0x1C. This caused a false "ERROR" banner on legitimate packets
(confirmed on a real KD3DKY-7 packet whose speed/course byte was
0x23, a normal digit-7 encoding that had already been iGated
successfully by the real decoder).

Fix: correct the range floor to 0x1C. Add a dedicated, always-on
check for 0x20 (SPACE) specifically in the 3 longitude bytes,
since it's spec-reserved regardless of numeric range. Stop
flagging 0x7f (DEL) from this heuristic entirely, since the
inspector has no way to see the destination's offset bit and
therefore can't know whether a given DEL byte is dangerous.

Updated the stale test comment referencing the old 0x26-0x7F
range and added three new packetInspect.test.js cases: a low
valid digit byte (0x23, no issues), a SPACE byte in the longitude
field (flagged as error), and a DEL byte in the longitude field
(not flagged, since offset can't be determined here).

Also fixed an unrelated mangled comment in mice.go where a smart-quote autocorrect had corrupted ''' or '' into '\” or ''.

Verified: go build ./..., go vet ./pkg/aprs/..., go test ./pkg/aprs/... (all pass including new cases), and node --test
web/src/lib/packetInspect.test.js (18/18 pass).

@kotfu

kotfu commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Here's (I think) the packets from discord:

2026-09-07 22:55:55 EDT: KE0OSV-9>SXUQ8S,WIDE1-1,qAR,N7JYS-3:`~QRrRuj/`":&}_6
2026-09-07 23:00:55 EDT: KE0OSV-9>SXUQ4Q,qAR,N7JYS-3:`~JQrRnj/`":)}_6

This is the beginning of the long discord discussion on this topic: https://discord.com/channels/1494724418868609026/1494724418868609029/1546202749132218458

@kotfu kotfu added the bug Something isn't working label Sep 10, 2026
@kotfu

kotfu commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Thanks for this — carrying the literal on-air bytes into the tests is what made it reviewable.

I verified the core and it holds: lonOffset is only ever 0 or 100, the NX0R-7 vectors decode to Hays KS, and 0x1C is the right floor (byte = digit + 28). Both suites pass.

Two things before merge.

1. The offset=100 DEL rejection is probably the same false positive

Running the current decoder math on the two vectors that check exists for:

DL9DAK {0x7f,'U','h'}  -> d=199 -> d -= 190 -> 9 -> 9.9627E
DL8XI  {0x7f,'(',0x7f} -> d=199 -> d -= 190 -> 9 -> 9.2165E

Both land inside 0..179, and both are plausible for DL-prefix stations. #76 (2026-05-05) justified the check as "199 deg ... plots a German station off the Aleutians" — true then, but #219 (ae6ebf3, 2026-06-10) reordered offset-before-fixups so APRS101's 190..199 rule applies and the wrap no longer happens.

So the new comment's claim that DEL+offset "yields a value outside the 0..179° range" is arithmetically false — and it points the next reader the wrong way, implying the range guard already covers this, which invites deleting the check and reintroducing #76.

Either direction is defensible: drop the offset=100 branch too, or keep it and rewrite the comment to say what actually justifies it (pre-GPS-lock sentinel, not out-of-range). It just can't stay as written. TestParseMicEDelInLonRejected's header has the same "~-161°" claim and needs the same treatment.

2. The inspector can see the offset bit

validateMicE already takes result, result.dest is populated at packetInspect.js:134, and result is currently unused in the function body. The offset is just the high-bit test on dest.raw[4] (A-J, K, L, P-Y, Z), so the "can't see it" justification doesn't hold.

Consequence: analyzeFrame on the DL9DAK frame returns issues: [] for a packet the decoder rejects outright — the one case the inspector most needs to explain.

Smaller

  • Lowering the floor to 0x1C also drops SPACE flagging for the speed/course bytes; the new loop only covers offsets 1-3. Verified SPACE at offset 5 now yields no issues. accept_broken_mice keys off body[4] == ' ' as a known repairable shape, so that flag is worth keeping.
  • The two loops can both fire, so one packet can emit two overlapping errors. The old single loop emitted at most one.
  • frame('T0TR4P', 'KD3DKY-7', ...): addr() truncates to 6 chars, so the fixture actually encodes KD3DKY-0.
  • The relaxed loop also newly admits DEL in the minutes and hundredths bytes at offset=0; neither is covered. The DL8XI bytes would give you a cheap case.

Requested changes

  1. Resolve the offset=100 DEL question, and make the comment match — in mice.go and in the test header.
  2. Either derive the offset in validateMicE and flag DEL when it's 100, or drop the claim that it can't be seen.
  3. Restore SPACE flagging for info offsets 4-6.
  4. Fold the two validateMicE loops into one.
  5. Fix the KD3DKY SSID fixture; add an offset=0 case with DEL outside the degrees byte.

@kotfu kotfu added the in-progress Fix or feature implementation in progress. label Sep 14, 2026
Two independent false-positive rejections were dropping/flagging valid
Mic-E packets, discovered while diagnosing why NX0R-7's beacons were
never gated to APRS-IS despite decoding fine on other iGates.

pkg/aprs/mice.go (decodeMicELon):
  decodeMicELon rejected 0x7f (DEL) unconditionally in any of the 3
  Mic-E longitude bytes as an "ambiguous/no GPS fix" sentinel. That
  check was added (issue chrissnell#76) to guard against a specific wraparound
  danger (DL9DAK/DL8XI reports) that only occurs when DEL combines
  with the destination's +100 degree longitude-offset bit. When the
  offset bit is NOT set, 0x7f is just the ordinary top-of-range
  digit-99 encoding (byte = digit + 28) and decodes to a real,
  plausible position. NX0R-7's on-air packets have offset=0, so this
  unconditional check was silently dropping every one of its beacons
  before they ever reached the RF->IS gate.

  Fix: keep 0x20 (SPACE) always rejected in the longitude field -
  it's the APRS101 ch 10 spec-reserved "unknown data" sentinel and
  has no offset-dependent nuance. Reject 0x7f (DEL) only when the
  destination's longitude offset is actually 100, matching the exact
  real-world bug pattern the check was written for.

  Added TestParseMicEDelInLonAcceptedNoOffset using the real
  NX0R-7 packet bytes (dest S8UR7Y and S8UV2X, offset=0) asserting
  they now decode to sane coordinates instead of erroring. Existing
  TestParseMicEDelInLonRejected (DL9DAK/DL8XI, offset=100) and
  TestParseMicEAmbiguousLonRejected (SPACE, offset=100) continue to
  pass unchanged.

web/src/lib/packetInspect.js (validateMicE):
  The web packet inspector's Mic-E byte-range sanity check used
  0x26-0x7F as the "encodable range", which is wrong - the real
  encoding (byte = digit + 28) legally produces bytes as low as
  0x1C. This caused a false "ERROR" banner on legitimate packets
  (confirmed on a real KD3DKY-7 packet whose speed/course byte was
  0x23, a normal digit-7 encoding that had already been iGated
  successfully by the real decoder).

  Fix: correct the range floor to 0x1C. Add a dedicated, always-on
  check for 0x20 (SPACE) specifically in the 3 longitude bytes,
  since it's spec-reserved regardless of numeric range. Stop
  flagging 0x7f (DEL) from this heuristic entirely, since the
  inspector has no way to see the destination's offset bit and
  therefore can't know whether a given DEL byte is dangerous.

  Updated the stale test comment referencing the old 0x26-0x7F
  range and added three new packetInspect.test.js cases: a low
  valid digit byte (0x23, no issues), a SPACE byte in the longitude
  field (flagged as error), and a DEL byte in the longitude field
  (not flagged, since offset can't be determined here).

Also fixed an unrelated mangled comment in mice.go where a smart-quote
autocorrect had corrupted '\'' or '`' into '\” or '`'.

Verified: go build ./..., go vet ./pkg/aprs/..., go test ./pkg/aprs/...
(all pass including new cases), and node --test
web/src/lib/packetInspect.test.js (18/18 pass).
Review of e7a29ab found that decodeMicELon's special-case rejection of
0x7f (DEL) combined with the destination's +100 deg longitude offset bit
no longer matches the arithmetic it claims to guard against. Issue chrissnell#76
justified the rejection on the premise that DEL + offset wraps to
~-161 deg (off Alaska), but issue chrissnell#219 (ae6ebf3) later reordered
decodeMicELon to add the +100 deg offset *before* the 180..189/190..199
wrap normalisation. Under that order, DEL (raw digit 99) + offset 100
always lands at raw degrees 199, which normalises to a valid 9 deg --
never out of range. The rejection was therefore a false positive with
a comment that no longer described reality.

- pkg/aprs/mice.go: remove the offset==100 DEL branch entirely; DEL is
  now always decoded as an ordinary top-of-range digit, matching how
  offset=0 DEL packets (NX0R-7) were already handled. Rewrote the
  surrounding comments and the ErrMicELonAmbiguous doc to drop the
  disproven "-161 deg / off Alaska" claim and state the real reasoning.
  Also fixed an unrelated mangled comment on parseMicE (a stray
  smart-quote had corrupted the '\'' / '`' type-byte reference).
- pkg/aprs/mice_test.go: replaced TestParseMicEDelInLonRejected (which
  asserted these DL9DAK/DL8XI packets error out) with
  TestParseMicEDelInLonAcceptedWithOffset, asserting they now decode
  to their real positions (~53.60N/9.96E and ~53.64N/9.22E, both near
  Hamburg, DE) -- consistent with the corrected arithmetic.
- web/src/lib/packetInspect.js: the inspector's validateMicE had the
  same stale "DEL is dangerous combined with the offset bit, which
  this check can't see" reasoning; removed it since DEL is never
  flagged now, matching the decoder. Folded the range-check and
  SPACE-check loops (which covered different byte ranges and could
  both fire on one packet) into a single loop over info offsets 1-6:
  SPACE in the longitude bytes (1-3) stays an 'error' (position is
  unplottable), SPACE in the speed/course bytes (4-6) is newly
  flagged as a 'warn' (position still decodes; this is also the shape
  accept_broken_mice repairs) -- previously silently dropped after an
  earlier fix lowered the encodable-byte floor to 0x1C.
- web/src/lib/packetInspect.test.js: fixed a broken test fixture where
  frame('T0TR4P', 'KD3DKY-7', ...) silently encoded SSID 0 instead of
  7 (addr() takes SSID as a separate argument and never parses a '-N'
  suffix out of a bare callsign string); added coverage for DEL in the
  longitude minutes/hundredths bytes (not just degrees) and for the
  new speed/course SPACE warning.

All existing and new tests pass: go test ./pkg/aprs/... and
node --test web/src/lib/packetInspect.test.js.
@zwolanj
zwolanj force-pushed the bugfix/mic-e-decoding-fix branch from 665efb5 to d0f77cd Compare September 20, 2026 15:35
@zwolanj

zwolanj commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

I updated the merge request with the changes you requested

kotfu and others added 3 commits September 23, 2026 18:25
# Conflicts:
#	web/src/lib/packetInspect.js
#	web/src/lib/packetInspect.test.js
An editor autocorrect turned the '\'' literal in the parseMicE doc
comment into a Unicode right double quote. Restore the ASCII form
that main had.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the speed/course bytes 0x20 is the ordinary digit 4, and the Go
decoder reads it normally: chrissnell#596's F4MLV-7 frame (6c 20 60) is 0 kt /
68 degrees. Warning on any SPACE there, with text claiming speed/course
decodes as unavailable, was a false positive, and it broke the chrissnell#596
regression test once merged with main.

Keep the SPACE error for the longitude bytes, where it is the APRS101
"unknown data" sentinel. For speed/course, warn only on the exact
accept_broken_mice shape the decoder realigns (lone space at offset 5,
symbol table one byte early), via an isMicESymTable helper that
matches pkg/aprs/mice.go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kotfu
kotfu merged commit 9e71676 into chrissnell:main Sep 24, 2026
7 checks passed
@kotfu

kotfu commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

Thanks @zwolanj -- you worked through every point in the review, and carrying the real NX0R-7, DL9DAK and DL8XI bytes into the tests made this easy to verify. The Go side is exactly right: dropping the DEL rejection entirely is the correct call now that #219 folds offset+199 back to 9 degrees, and the rewritten comments and test headers explain why.

I pushed a few commits onto your branch to get it merged:

  1. Merged main. web: accept Mic-E offset bytes down to 0x1c in packet inspector #599 (from Packet Inspector rejects valid Mic-E bytes below 0x26 #596) landed the same 0x1C inspector floor in the meantime, so packetInspect.js and its tests conflicted. I kept your inspector code and kept both sets of tests.
  2. Restored the quote in mice.go. The parseMicE comment on main already had the correct '\''. The change on the branch had turned it into a Unicode right double quote (probably editor autocorrect), so I put the ASCII back.
  3. Narrowed the speed/course SPACE warning. This one was my mistake, not yours. My review asked you to restore SPACE flagging for offsets 4-6, but in the speed/course bytes 0x20 is just digit 4 and decodeMicESpeedCourse reads it normally. Packet Inspector rejects valid Mic-E bytes below 0x26 #596's F4MLV-7 frame (6c 20 60, 0 kt / 68 degrees) is a legitimate example, and the warning fired on it and broke web: accept Mic-E offset bytes down to 0x1c in packet inspector #599's regression test. The inspector now only warns when the bytes match the exact shape accept_broken_mice repairs (a lone space at offset 5 with the symbol table one byte early), using an isMicESymTable helper that mirrors the Go one. SPACE in the longitude bytes is still an error.

Web tests (503) and go test ./pkg/aprs/... pass, and CI was green on the combined branch before merge. Thanks again for chasing down why NX0R-7 wasn't being gated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working in-progress Fix or feature implementation in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants