diff --git a/ROADMAP.md b/ROADMAP.md index a372d18..afab51d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -11,30 +11,35 @@ and ideas that are not ready for scheduling are recorded in ## Active milestone -### Milestone 24.8 — MSI documented menu projection foundation - -The first three Milestone 24.8 slices established lossless bounded MSI XML -modeling/parsing, exact serialized `MSI\r` retrieval for the existing -CR-line/replay path, and the six indexed `MNU` forms whose menu/index roles are -shared by the official SDS100/SDS200 Remote Command Specification V1.02 -(2023-12-22) and SDS Series Remote Command Specification V2.00 (2025-07-07). - -This fourth narrow slice adds a read-only documented projection over the -existing lossless `MsiResponse`. Both specifications define the same MSI root -attributes (`Name`, `Index`, `MenuType`, `Value`, `Selected`) and the same -`MenuItem`, `MenuInput`, `MenuLocation`, and `MenuErrorMsg` attribute names. -Typed projection records expose those named fields while retaining their exact -string values and complete attribute mappings. Missing, unknown, repeated, and -future values continue to be preserved rather than rejected or coerced. - -The original root attributes, all descendant `MsiRecord` values, source order, -unknown elements/attributes, and raw XML remain authoritative and unchanged. -This slice sends no new scanner command and adds no `MSV`/`MSB` execution, -unindexed `MNU`, menu lifecycle/state ownership, renderer behavior, UDP/XML-map -support, model/firmware applicability, or physical scanner claim. In particular, -the serialized value of the documented `[RSV]` field in `MSV`/`MSB` remains -unresolved and is not inferred from unrelated empty-parameter conventions. -Quick Search (`QSH`) likewise remains blocked pending exact `FRQ` evidence. +### Milestone 24.8 — MSI direct-UDP retrieval foundation + +The first four Milestone 24.8 slices established lossless bounded MSI XML +modeling/parsing, exact serialized `MSI\r` retrieval for CR-line/replay +transports, the six evidence-backed indexed `MNU` forms, and a read-only typed +projection for the documented MSI menu fields while preserving all lossless +source evidence. + +This fifth narrow slice promotes exact one-shot `MSI\r` retrieval into the +existing SDS200 UDP bounded-XML machinery. The shared production XML command +map now includes `MSI -> MSI`; an exact bare `MSI` request establishes a +one-shot MSI expectation; matching bare or explicitly prefixed MSI XML can be +assembled; numbered `Footer`/`Foot` fragments use the existing bounded +reassembly path; retryable sequence failures resend the exact original `MSI` +wire; and successful completion clears one-shot MSI retry/expectation state. + +Direct `UdpTransport` and its capture wrapper are covered only with deterministic +fake datagrams. This is software transport/framing evidence, not a physical +scanner or firmware-support claim. Fallback transports remain blocked because +their active candidate can change and no fallback-wide MSI transport contract +has been established. Custom transports that merely advertise an `udp://` +endpoint remain fail-closed unless they are the repository's direct +`UdpTransport` path. + +This slice does not add `MSV`/`MSB` execution, unindexed `MNU`, menu +lifecycle/state ownership, renderer behavior, model/firmware applicability, or +physical validation. The serialized `[RSV]` field in `MSV`/`MSB` remains +unresolved, and Quick Search (`QSH`) remains blocked pending exact `FRQ` +evidence. ## Deferred hardware validation @@ -420,14 +425,15 @@ Tentative evidence-led slicing is: Milestone 24.8 began with a receive-only `MSI` bounded-XML model/parser foundation preserving the exact reviewed `` root, root attributes, ordered/repeated descendants, unknown attributes and elements, and raw XML. The -second narrow slice adds exact `MSI` retrieval through typed command/radio APIs -and radio-local bounded-XML assembly for CR-line serial and deterministic replay -coverage. The shared production XML command map remains unchanged, and the -serialized MSI request/response path rejects direct/capture-wrapped UDP and -fallback transports before transmission. UDP expectation/retry/bare-XML and -fallback MSI behavior therefore remain deferred. No menu-field semantics, -`MNU`/`MSV`/`MSB` controls, menu lifecycle, or model/firmware/physical-scanner -applicability is inferred. +second slice added exact `MSI` retrieval through typed command/radio APIs for +CR-line serial and deterministic replay. The third added only the six indexed +`MNU` rows whose index roles are explicit in both reviewed SDS specifications, +and the fourth added a read-only documented MSI menu projection without +discarding lossless source evidence. The fifth slice promotes exact one-shot +`MSI` retrieval to the shared SDS200 UDP bounded-XML expectation, fragment +reassembly, and retry machinery with deterministic fake datagrams. Fallback +MSI remains blocked, and no `MSV`/`MSB`, unindexed `MNU`, menu lifecycle, +model/firmware applicability, or physical-scanner behavior is inferred. Milestone 24.1 selected `GLT` as the safest leading implementation candidate because it is bounded retrieval that complements the Favorites foundation and diff --git a/docs/advanced-protocol-research.md b/docs/advanced-protocol-research.md index 76c8003..6784db1 100644 --- a/docs/advanced-protocol-research.md +++ b/docs/advanced-protocol-research.md @@ -326,11 +326,12 @@ existing behavior. ### SDS200 UDP XML reconstruction `UdpDatagramDecoder` reconstructs numbered XML fragments using `Foot`/`Footer`, -`No`, and `EOT` after a command is recognized. Milestone 24.2 extends command +`No`, and `EOT` after a command is recognized. Milestone 24.2 extended command expectation and retry bookkeeping from the existing GSI/PSI paths to exact `GLT,FL`, while reconstruction remains a transport concern and GLT domain -semantics remain above `network.py`. MSI, AST, and other future protocols are -still deferred. +semantics remain above `network.py`. Milestone 24.8's fifth slice now reuses +that same one-shot bounded-XML machinery for exact `MSI`, without changing MSI +domain parsing or claiming physical UDP support. ### Existing stream lifecycle @@ -575,10 +576,27 @@ structural test data, not claims about physically observed menu indexes, values, limits, keys, or scanner states. `MSV` and `MSB` execution remains blocked: V1.02 and V2.00 document their outer forms but do not establish the serialized value of `[RSV]`, and the reviewed official RH-536HP predecessor source contains -no searchable MSV/MSB implementation that resolves it. Unindexed MNU, menu -lifecycle/state ownership, renderer exposure, UDP/XML-map behavior, -model/firmware applicability, physical transport applicability, and physical -scanner validation likewise remain deferred. +no searchable MSV/MSB implementation that resolves it. + +The fifth narrow slice promotes exact `MSI` into `XML_COMMAND_ROOTS` and the +existing SDS200 direct-UDP one-shot bounded-XML machinery. An exact bare `MSI` +request establishes the expectation; a nonexact `MSI,` or `MSI,...` request does +not. Matching bare or explicitly prefixed MSI XML is correlated by root, +numbered `Footer`/`Foot` fragments reuse the existing ordered reassembly path, +retryable sequence failures resend the exact original `MSI` wire, and successful +completion clears MSI one-shot retry state. `SDSScanner.get_msi()` is therefore +allowed on the repository's direct `UdpTransport`, including when wrapped by +capture recording. + +All UDP evidence in this slice is deterministic fake-datagram software evidence. +It does not establish physical SDS200 behavior, firmware availability, or a +broader transport guarantee. Fallback transports remain fail-closed because +their active transport can change and no fallback-wide MSI framing contract is +established. Custom controls that merely present an `udp://` endpoint are not +treated as evidence-equivalent to `UdpTransport`. Unindexed MNU, menu +lifecycle/state ownership, renderer exposure, model/firmware applicability, +physical transport applicability, and physical scanner validation remain +deferred. ### Richer NAC, RAN, color-code, area, activity, and quality data diff --git a/src/sds200/network.py b/src/sds200/network.py index f61967a..1990b7f 100644 --- a/src/sds200/network.py +++ b/src/sds200/network.py @@ -141,7 +141,7 @@ def reset(self) -> None: def expect_command(self, command: str) -> None: """Record commands whose UDP response may be a bare XML document.""" normalized = command.rstrip("\r\n").strip() - name, _, argument = normalized.partition(",") + name, separator, argument = normalized.partition(",") name = name.upper() with self._lock: @@ -158,6 +158,8 @@ def expect_command(self, command: str) -> None: self._stream_xml_command = "PSI" elif name == "GLT" and argument.strip().upper() == "FL": self._expected_xml_command = "GLT" + elif name == "MSI" and not separator: + self._expected_xml_command = "MSI" def feed(self, data: bytes) -> tuple[str, ...]: text = data.decode("utf-8", errors="replace").strip("\x00") @@ -487,7 +489,7 @@ def write_command(self, command: str) -> None: self._send_normalized(normalized, retry=False) def _remember_xml_command(self, normalized: str) -> None: - command, _, argument = normalized.partition(",") + command, separator, argument = normalized.partition(",") command = command.upper() self._decoder.expect_command(normalized) if command == "GSI": @@ -500,7 +502,9 @@ def _remember_xml_command(self, normalized: str) -> None: else: self._last_xml_commands[command] = normalized self._xml_retry_counts[command] = 0 - elif command == "GLT" and argument.strip().upper() == "FL": + elif ( + command == "GLT" and argument.strip().upper() == "FL" + ) or (command == "MSI" and not separator): self._last_xml_commands[command] = normalized self._xml_retry_counts[command] = 0 @@ -642,7 +646,7 @@ def _handle_decoder_diagnostic(self, diagnostic: TransportDiagnostic) -> None: def _xml_completed(self, command: str) -> None: self._xml_retry_counts[command] = 0 - if command == "GLT": + if command in {"GLT", "MSI"}: self._last_xml_commands.pop(command, None) self._xml_retry_counts.pop(command, None) with self._statistics_lock: diff --git a/src/sds200/radio.py b/src/sds200/radio.py index c7c9150..5f1593d 100644 --- a/src/sds200/radio.py +++ b/src/sds200/radio.py @@ -114,7 +114,6 @@ WaterfallSubscription, ) from .xml_protocol import ( - XML_COMMAND_ROOTS, AnalysisParser, GltParser, MsiParser, @@ -194,6 +193,7 @@ def __init__( fallback_transport = ( self.transport if isinstance(self.transport, FallbackTransport) else None ) + direct_udp_msi_supported = isinstance(self.transport, UdpTransport) if capture_path is not None: self.transport = RecordingTransport( self.transport, @@ -206,7 +206,7 @@ def __init__( self.glt_parser = GltParser() self.analysis_parser = AnalysisParser() self.msi_parser = MsiParser() - self.xml_assembler = XmlResponseAssembler({**XML_COMMAND_ROOTS, "MSI": "MSI"}) + self.xml_assembler = XmlResponseAssembler() self.events = EventBus() self._analysis_publisher = AnalysisPublisher() self._waterfall_publisher = WaterfallPublisher() @@ -218,6 +218,7 @@ def __init__( self._response_lock = threading.RLock() self._command_lock = threading.RLock() self._fallback_transport = fallback_transport + self._direct_udp_msi_supported = direct_udp_msi_supported if self._fallback_transport is not None: self._fallback_transport.set_recovery_guard(self._recovery_idle) self._closed = threading.Event() @@ -1161,12 +1162,15 @@ def _wait_for_response( ) if response_command == "MSI" and ( - self.endpoint.startswith("udp://") - or self._fallback_transport is not None + self._fallback_transport is not None + or ( + self.endpoint.startswith("udp://") + and not self._direct_udp_msi_supported + ) ): raise UnsupportedScannerFeatureError( - "MSI retrieval is unavailable on UDP and fallback " - "control transports." + "MSI retrieval is unavailable on unverified UDP-like and " + "fallback control transports." ) response_queue: queue.Queue[object] = queue.Queue(maxsize=1) diff --git a/src/sds200/xml_protocol.py b/src/sds200/xml_protocol.py index afaa830..eebbd48 100644 --- a/src/sds200/xml_protocol.py +++ b/src/sds200/xml_protocol.py @@ -17,7 +17,13 @@ ) XML_COMMAND_ROOTS: Mapping[str, str] = MappingProxyType( - {"GSI": "ScannerInfo", "PSI": "ScannerInfo", "GLT": "GLT", "AST": "AST"} + { + "GSI": "ScannerInfo", + "PSI": "ScannerInfo", + "GLT": "GLT", + "AST": "AST", + "MSI": "MSI", + } ) diff --git a/tests/fixtures/advanced_protocol/README.md b/tests/fixtures/advanced_protocol/README.md index 8acb7f5..150922d 100644 --- a/tests/fixtures/advanced_protocol/README.md +++ b/tests/fixtures/advanced_protocol/README.md @@ -80,3 +80,11 @@ unverified unless separately documented. command beyond existing MSI retrieval and does not establish MSV/MSB reserved field serialization, menu mutation/lifecycle semantics, UDP support, model/firmware applicability, renderer exposure, or physical validation. +- The fifth narrow Milestone 24.8 slice adds no new fixture file. Instead, + deterministic fake SDS200 UDP datagrams reuse the existing synthetic MSI + shapes to validate software-only one-shot expectation, bare/prefixed root + correlation, numbered fragment reassembly, exact-wire retry, capture-wrapper + behavior, and state-neutral typed retrieval. This does not convert synthetic + transport tests into physical scanner, firmware, or model evidence. Fallback + MSI, unindexed MNU, MSV/MSB execution, menu lifecycle, and renderer behavior + remain outside the slice. diff --git a/tests/test_network.py b/tests/test_network.py index 5cf61f7..fdf25e4 100644 --- a/tests/test_network.py +++ b/tests/test_network.py @@ -5,6 +5,7 @@ import time import xml.etree.ElementTree as ET from collections.abc import Callable +from pathlib import Path import pytest @@ -263,15 +264,24 @@ def test_decoder_wraps_bare_glt_xml_once_after_exact_command() -> None: assert decoder.feed(bare_xml) == (bare_xml.decode(),) -def test_udp_decoder_does_not_register_or_wrap_msi_xml() -> None: - assert "MSI" not in XML_COMMAND_ROOTS +def test_udp_decoder_wraps_bare_msi_xml_once_after_exact_command() -> None: + assert XML_COMMAND_ROOTS["MSI"] == "MSI" decoder = UdpDatagramDecoder() decoder.expect_command("MSI") bare_xml = b'' + assert decoder.feed(bare_xml) == ("MSI,,", bare_xml.decode()) + assert decoder.feed(bare_xml) == (bare_xml.decode(),) + + +@pytest.mark.parametrize("command", ["MSI,", "MSI,FUTURE"]) +def test_udp_decoder_does_not_expect_nonexact_msi_command(command: str) -> None: + decoder = UdpDatagramDecoder() + decoder.expect_command(command) + bare_xml = b'' + assert decoder.feed(bare_xml) == (bare_xml.decode(),) - assert decoder.feed(b"MSI,," + bare_xml) == () def test_decoder_malformed_bare_glt_does_not_complete_expectation() -> None: @@ -347,6 +357,33 @@ def test_decoder_reassembles_numbered_glt_datagrams_in_source_order() -> None: assert root[0].attrib["FutureAttr"] == "preserve-me" +def test_decoder_reassembles_numbered_msi_datagrams_in_source_order() -> None: + completed: list[str] = [] + decoder = UdpDatagramDecoder(completion_handler=completed.append) + decoder.expect_command("MSI") + first = ( + b'MSI,,' + b'' + b'