From 8f14b81197e40ae5b06eaac13649f58c200a1aa3 Mon Sep 17 00:00:00 2001 From: stevenboyd78 Date: Thu, 6 Aug 2026 03:35:04 -0600 Subject: [PATCH 1/5] Document daemon deployment and upgrades --- README.md | 15 +- ROADMAP.md | 111 ++++--------- docs/configuration.md | 16 +- docs/daemon-api.md | 6 +- docs/daemon-deployment.md | 341 ++++++++++++++++++++++++++++++++++++++ docs/daemon-events.md | 8 +- docs/daemon-runtime.md | 39 +++-- docs/logging.md | 35 ++-- scripts/check_docs.py | 1 + wiki/Home.md | 3 + wiki/Installation.md | 32 ++++ wiki/_Sidebar.md | 1 + 12 files changed, 488 insertions(+), 120 deletions(-) create mode 100644 docs/daemon-deployment.md diff --git a/README.md b/README.md index 6e3751b..c8ce6e0 100644 --- a/README.md +++ b/README.md @@ -316,7 +316,7 @@ subscription is created. Frames preserve RTP sequence, timestamp, SSRC, continuity estimates, observation time, endpoint, raw payload bytes, and cumulative loss caused by that client's bounded queue. -Stop the process with `Ctrl+C` or `SIGTERM`. Shutdown first closes API clients, +Stop the process with `Ctrl+C` or `SIGTERM`. Shutdown first closes API clients, then stops daemon-owned destinations, then stops scanner, PSI, audio, and router ownership, closes PCMU clients, and finally closes event clients after final lifecycle transitions. All three owned sockets are removed. @@ -355,9 +355,15 @@ a valid 53.120-second 8 kHz mono WAV with an adjacent metadata sidecar. Quitting the TUI left scanner, PSI, audio, router, and daemon ownership healthy. A later controlled `SIGTERM` removed `daemon.sock`, `events.sock`, and `pcmu.sock`. -Decoded-PCM subscriptions, automatic daemon selection, and destination -activation remain follow-on work. The initial daemon router has no attached -destinations. See the +The daemon loads the explicit `--destination-config` path or +`${XDG_CONFIG_HOME:-~/.config}/sdsctl/daemon-destinations.toml` before opening +scanner hardware. Saved playback, recording, and remote-profile destinations are +activated under daemon ownership. `SIGHUP` transactionally reloads that exact +manifest while preserving the previous committed set on failure. + +Decoded-PCM subscriptions and automatic daemon selection remain follow-on work. +See the +[daemon deployment and upgrade guide](docs/daemon-deployment.md), [daemon runtime and process guide](docs/daemon-runtime.md), [local daemon API guide](docs/daemon-api.md), [local daemon event stream guide](docs/daemon-events.md), @@ -707,6 +713,7 @@ See [SECURITY.md](SECURITY.md) for vulnerability reporting and - [Reliability and observability](docs/reliability.md) - [Operational logging](docs/logging.md) - [Textual TUI](docs/tui.md) +- [Daemon deployment and upgrades](docs/daemon-deployment.md) - [Foreground daemon and ownership runtime](docs/daemon-runtime.md) - [Local daemon API](docs/daemon-api.md) - [Local daemon event stream](docs/daemon-events.md) diff --git a/ROADMAP.md b/ROADMAP.md index 2cdb4be..31c9c09 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -9,50 +9,28 @@ The broader product direction, architectural constraints, deferred capabilities, and ideas that are not ready for scheduling are recorded in [the project vision](docs/project-vision.md). -## Recently completed milestone +## Active milestone -### Milestone 19.8 — Safe daemon controls +### Milestone 19.12 — v0.19.0 release -- **Capability-checked control operations — implemented** - - Added explicit version 1 daemon operations for hold, next, previous, and - reconnect without unrestricted raw scanner-command passthrough. - - Reused typed scanner navigation contracts, model capability checks, strict - targets, bounded counts, and structured scanner acknowledgements. - - Preserved every read-only operation and standalone CLI/TUI behavior. - - Did not add resume because no documented or verified resume/unhold scanner - wire contract exists. -- **Serialized execution and authoritative completion — implemented** - - Added one nonblocking daemon mutation slot so conflicting clients receive - `control_busy` instead of interleaving or building a queue. - - Successful responses follow scanner acknowledgement and include an ordered - `DaemonControlResult` with timestamps and an authoritative runtime snapshot. - - Applied one maximum two-second request budget to runtime-lock acquisition and - scanner completion. - - Raised the default API worker shutdown deadline to three seconds and added - rejection of configurations that cannot outlast the maximum request duration. - - Limited daemon reconnect to the directly owned bounded SDS200 UDP transport; - serial, fallback, replay, and injected transports return - `unsupported_operation`. -- **Safety, isolation, and observability — implemented** - - Added structured redacted `control_busy`, `control_unavailable`, - `unsupported_operation`, `control_timeout`, `control_rejected`, and - `control_failed` responses. - - Kept resulting scanner and runtime changes observable through authoritative - snapshots and the existing ordered event stream. - - Preserved private socket permissions, bounded clients, worker isolation, and - the single-owner scanner lifecycle. -- **Regression, documentation, and hardware validation — complete** - - Added regression coverage for success, strict parameters, scanner rejection, - timeouts, unsupported transports, concurrent controls, shutdown, and - unchanged read-only behavior. - - Documented envelopes, completion semantics, deadlines, exclusions, - compatibility, and client responsibilities. - - Physically validated TGID hold, next, previous, hold release, and bounded - reconnect against an SDS200 while API, event, PSI, RTSP/RTP, decoded-audio, - and two PCMU clients remained healthy. - - Confirmed ordered control sequences, reversible hold state, reconnect - connection transitions, loss-free matching PCMU delivery, controlled - `SIGTERM`, successful process exit, and removal of all owned sockets. +- **Compatibility, migration, deployment, and systemd documentation — active** + - Document preserved distribution, import-package, executable, and legacy + profile compatibility. + - Document explicit destination manifests, service accounts, private sockets, + `SIGHUP` reload, systemd operation, upgrades, and rollback. +- **Acceptance validation — pending** + - Validate multiple clients, slow and malformed clients, shutdown fault + injection, clean installation, and upgrade behavior. + - Run the full static, test, documentation, packaging, and distribution suite. + - Complete physical SDS200 daemon-owned CLI and TUI client validation. +- **Publication — pending** + - Merge release preparation only after CI and CodeQL pass. + - Publish `v0.19.0` through trusted PyPI publishing and create the GitHub + release. + - Verify a clean installation from public PyPI. + +Keep the existing Python distribution and import package compatible until a +separate migration plan justifies a rename. ## Deferred hardware validation @@ -77,42 +55,6 @@ fixture-tested, not hardware-validated. These milestone groups preserve intended future work. Their numbering and release assignment may change before implementation begins. -### Milestone 19.9 — CLI daemon client - -- Add daemon status, snapshot, event-watch, safe-control, and optional audio - client workflows. -- Preserve explicit daemon and standalone selection with clear absent, - incompatible, and disconnected daemon diagnostics. -- Use the CLI migration to validate protocol compatibility before TUI adoption. - -### Milestone 19.10 — TUI daemon client - -- Consume daemon snapshots, ordered events, controls, and daemon-owned audio. -- Preserve an explicit standalone mode and show daemon protocol, connection, - reconnect, and degraded-state information. -- Ensure closing or reconnecting the TUI never stops the daemon-owned scanner - session. - -### Milestone 19.11 — Destination activation and reload - -- Activate saved playback, recording, and remote-stream destinations under daemon - ownership. -- Define validated, previewable configuration replacement and failure-isolated - destination updates. -- Use `SIGHUP` to load, validate, and transactionally replace the selected - destination manifest while preserving the previous committed set on failure. - -### Milestone 19.12 — v0.19.0 release - -- Complete compatibility, migration, deployment, and systemd documentation. -- Validate multiple clients, slow and malformed clients, shutdown fault - injection, clean installation, and upgrade behavior. -- Run full Python 3.11–3.14 CI and CodeQL validation plus physical SDS200 - daemon-owned CLI and TUI client testing. - -Keep the existing Python import package compatible until a separate migration -plan justifies a rename. - ### Milestone 20 — Web dashboard and Home Assistant - Add a responsive browser dashboard backed by the daemon API. @@ -305,3 +247,16 @@ fixtures before renderer-specific implementation. helpers, documentation, extensive regression coverage, a reusable hardware validator, and physical SDS200 validation with simultaneous API, event, and dual-PCMU clients. +- Milestone 19.8: capability-checked hold, next, previous, and bounded reconnect + controls; serialized mutation ownership; scanner-acknowledged completion; + stable redacted failures; regression coverage; and physical SDS200 validation. +- Milestone 19.9: explicit daemon CLI status, snapshots, safe controls, ordered + event watching, PCMU playback and WAV recording, protocol compatibility, and + physical SDS200 validation. +- Milestone 19.10: explicit daemon-backed TUI state, events, controls, playback, + recording, and saved-recording workflows without opening scanner hardware or + stopping daemon ownership, plus physical SDS200 validation. +- Milestone 19.11: validated playback, recording, and remote-profile destination + manifests; deterministic activation resources; transactional replacement; + failure-isolated reload; daemon lifecycle ownership; `SIGHUP`; regression + coverage; and physical SDS200 validation. diff --git a/docs/configuration.md b/docs/configuration.md index 32fe744..caf9725 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -34,7 +34,11 @@ The optional application configuration files are: - system: `/etc/sdsctl/config.toml`; - user: `${XDG_CONFIG_HOME:-~/.config}/sdsctl/config.toml`. -Path resolution also defines these locations for future persistent services: +The default daemon destination manifest is: + +- `${XDG_CONFIG_HOME:-~/.config}/sdsctl/daemon-destinations.toml`. + +Path resolution also defines persistent service locations: - state: `${XDG_STATE_HOME:-~/.local/state}/sdsctl/`; - cache: `${XDG_CACHE_HOME:-~/.cache}/sdsctl/`. @@ -154,6 +158,10 @@ The version 1 application schema contains no credential-bearing fields. Unknown fields are reported by name and source, not by value. Saved remote-audio destinations continue to store environment-variable secret -references rather than resolved passwords. Resolved credentials must not be -written to application configuration, logs, exceptions, traces, or serialized -output. +references rather than resolved passwords. The daemon destination manifest +selects those profiles by name and never stores resolved credentials. Resolved +credentials must not be written to application configuration, destination +configuration, logs, exceptions, traces, or serialized output. + +See [Daemon deployment and upgrade guide](daemon-deployment.md) for systemd, +destination-manifest, service-account, migration, and upgrade examples. diff --git a/docs/daemon-api.md b/docs/daemon-api.md index b8f755a..1bf05dc 100644 --- a/docs/daemon-api.md +++ b/docs/daemon-api.md @@ -436,5 +436,7 @@ Ordered events are available through their dedicated socket and `pcmu.sock` and `sdsctl daemon-client audio`. The daemon-backed TUI obtains its authoritative initial state and safe-control results through this API while ordered updates and PCMU audio remain on their dedicated sockets. Decoded-PCM -CLI workflows, discovery, and destination activation remain assigned to later -Milestone 19 work. +Automatic daemon discovery and decoded-PCM client workflows remain +follow-on work. Explicit CLI and TUI daemon clients, saved destination +activation, and transactional `SIGHUP` destination reload are part of the +current daemon contract. diff --git a/docs/daemon-deployment.md b/docs/daemon-deployment.md new file mode 100644 index 0000000..8c011d2 --- /dev/null +++ b/docs/daemon-deployment.md @@ -0,0 +1,341 @@ +# Daemon deployment and upgrade guide + +This guide covers installation, configuration, systemd operation, local-client +access, upgrades, and rollback for the foreground `sdsctl daemon` introduced +during Milestone 19. + +The daemon owns one SDS200 control connection, one PSI stream, one RTSP/RTP +audio session, one decoded-PCM router, three private local sockets, and any saved +playback, recording, or remote-stream destinations. It remains in the foreground +and is intended to be supervised by systemd or another process manager. + +## Compatibility and migration + +The v0.19.0 release preserves these existing names and files: + +- the Python distribution remains `sds200`; +- the import package remains `sds200`; +- the command remains `sdsctl`; +- saved scanner connection profiles remain under the legacy `sds200` + configuration root; +- saved remote-audio profiles remain under the legacy `sds200` + configuration root; and +- application, daemon, state, cache, and service paths use the `sdsctl` + namespace. + +No configuration file is moved, rewritten, or deleted automatically. Existing +standalone CLI, TUI, scanner-control, recording, and network-audio commands +remain available. Daemon-backed CLI and TUI operation is explicit. + +The relevant default paths are: + +| Purpose | Default path | +| --- | --- | +| System application configuration | `/etc/sdsctl/config.toml` | +| User application configuration | `${XDG_CONFIG_HOME:-~/.config}/sdsctl/config.toml` | +| Daemon destination manifest | `${XDG_CONFIG_HOME:-~/.config}/sdsctl/daemon-destinations.toml` | +| Legacy scanner profiles | `${XDG_CONFIG_HOME:-~/.config}/sds200/profiles.toml` | +| Legacy remote-audio profiles | `${XDG_CONFIG_HOME:-~/.config}/sds200/remote-audio-profiles.toml` | +| User state fallback | `${XDG_STATE_HOME:-~/.local/state}/sdsctl/` | +| User cache | `${XDG_CACHE_HOME:-~/.cache}/sdsctl/` | +| Runtime sockets with `XDG_RUNTIME_DIR` | `$XDG_RUNTIME_DIR/sdsctl/` | + +When `XDG_RUNTIME_DIR` is absent, the API, event, and PCMU sockets fall back to +the user-state directory. + +## Install into a dedicated virtual environment + +The following example installs the released package under `/opt/sdsctl`. Adapt +account-management commands to the local distribution. + +```bash +sudo useradd \ + --system \ + --create-home \ + --home-dir /var/lib/sdsctl \ + --shell /usr/sbin/nologin \ + sdsctl + +sudo python3 -m venv /opt/sdsctl +sudo /opt/sdsctl/bin/python -m pip install --upgrade pip +sudo /opt/sdsctl/bin/python -m pip install "sds200[tui,playback]" + +sudo install -d -o root -g root -m 0755 /etc/sdsctl +sudo install -d -o sdsctl -g sdsctl -m 0700 \ + /var/lib/sdsctl/recordings +``` + +Install only the feature groups the service needs. Recording and remote +streaming do not require local playback. Local playback requires an operating +system audio backend and appropriate device permissions. + +Verify the installed executable before creating the service: + +```bash +/opt/sdsctl/bin/sdsctl --version +/opt/sdsctl/bin/sdsctl daemon --help +``` + +## Application configuration + +Application-wide reconnect, presentation, and logging settings may be placed in +`/etc/sdsctl/config.toml`: + +```toml +version = 1 + +[application] +reconnect_attempts = 0 +reconnect_initial_delay = 1.0 +reconnect_multiplier = 2.0 +reconnect_max_delay = 30.0 +health_history_limit = 250 +log_level = "INFO" +``` + +System configuration has lower precedence than user configuration, environment +variables, and explicit CLI arguments. See +[Layered application configuration](configuration.md). + +## Destination manifest + +The daemon reads one strict version 1 destination manifest before opening scanner +hardware. An absent manifest means no daemon-owned destinations. A valid empty +manifest removes all active destinations during reload. + +A system-service deployment can select an explicit manifest at +`/etc/sdsctl/daemon-destinations.toml`: + +```toml +version = 1 + +[destinations.archive] +kind = "recording" +path = "/var/lib/sdsctl/recordings/live.wav" +overwrite = true +buffer_seconds = 5.0 + +[destinations.speakers] +kind = "playback" +backend = "auto" +buffer_ms = 250 + +[destinations.feed] +kind = "remote-profile" +profile = "county-feed" +publish_metadata = true +metadata_minimum_update_interval = 2.5 +``` + +Destination names must be unique. Recording paths must be absolute and their +parent directories must already exist. Supported playback backends are `auto`, +`sounddevice`, `pipewire`, `pulseaudio`, and `alsa`. The optional `device` +field is a PortAudio name or index for `sounddevice`, or the backend-specific +text device or target for command-backed adapters. + +A `remote-profile` destination resolves its named profile through the existing +legacy remote-audio profile store. For a service account whose home is +`/var/lib/sdsctl`, the default store is: + +```text +/var/lib/sdsctl/.config/sds200/remote-audio-profiles.toml +``` + +Remote profiles retain environment-variable references rather than resolved +credentials. Supply referenced secrets through a root-owned environment file: + +```bash +sudo install -o root -g root -m 0600 /dev/null /etc/sdsctl/sdsctl.env +``` + +Do not place resolved credentials in the destination manifest, application +configuration, unit file, logs, traces, or captures. + +## systemd service + +Create `/etc/systemd/system/sdsctl.service`: + +```ini +[Unit] +Description=SDS200 scanner ownership daemon +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=sdsctl +Group=sdsctl +Environment=PYTHONUNBUFFERED=1 +EnvironmentFile=-/etc/sdsctl/sdsctl.env +ExecStart=/opt/sdsctl/bin/sdsctl --log-level INFO --host 192.168.0.251 daemon --destination-config /etc/sdsctl/daemon-destinations.toml --socket-path /run/sdsctl/daemon.sock --event-socket-path /run/sdsctl/events.sock --pcmu-socket-path /run/sdsctl/pcmu.sock +ExecReload=/bin/kill -HUP $MAINPID +Restart=on-failure +RestartSec=5 +KillSignal=SIGTERM +TimeoutStopSec=20 +RuntimeDirectory=sdsctl +RuntimeDirectoryMode=0700 +StateDirectory=sdsctl +StateDirectoryMode=0700 +UMask=0077 +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true + +[Install] +WantedBy=multi-user.target +``` + +Global scanner and logging options must precede `daemon`; daemon-specific +options follow it. Replace the explicit host with a saved network-capable +profile when appropriate: + +```ini +ExecStart=/opt/sdsctl/bin/sdsctl --log-level INFO --profile home daemon --destination-config /etc/sdsctl/daemon-destinations.toml --socket-path /run/sdsctl/daemon.sock --event-socket-path /run/sdsctl/events.sock --pcmu-socket-path /run/sdsctl/pcmu.sock +``` + +A fallback profile may use serial control while retaining its SDS200 network host +for RTSP/RTP audio. Serial-only profiles, replay captures, and non-SDS200 +network-audio selections are rejected. + +`ProtectSystem=strict` leaves the system read-only except for locations managed +by systemd, including `RuntimeDirectory` and `StateDirectory`. Add narrowly +scoped writable paths only when a selected destination requires them. + +Local playback from a system service requires access to the selected audio +device. An ALSA deployment may require `SupplementaryGroups=audio`. A per-user +PulseAudio or PipeWire session may not be available to a system service; verify +the chosen backend with `sdsctl audio-devices` under the service account. + +Validate and start the unit: + +```bash +sudo systemd-analyze verify /etc/systemd/system/sdsctl.service +sudo systemctl daemon-reload +sudo systemctl enable --now sdsctl.service +sudo systemctl status sdsctl.service +``` + +Inspect logs with: + +```bash +journalctl -u sdsctl.service +journalctl -u sdsctl.service --since today +journalctl -u sdsctl.service -f +``` + +## Reload destinations + +`SIGHUP` reloads the exact manifest selected at startup. The daemon loads and +validates the replacement before beginning activation. + +```bash +sudo systemctl reload sdsctl.service +``` + +A successful reload transactionally replaces the committed destination set. +Load or activation failure leaves the prior committed destinations running. +Post-commit cleanup failure is reported without rolling back a successfully +activated replacement. `SIGTERM` remains the orderly stop signal. + +Before reloading, validate a manifest through the same loader without starting +scanner hardware: + +```bash +/opt/sdsctl/bin/python - <<'PY' +from sds200 import load_daemon_destination_configuration + +configuration = load_daemon_destination_configuration( + "/etc/sdsctl/daemon-destinations.toml" +) +print(configuration.as_dict()) +PY +``` + +## Local client access + +The daemon sockets use mode `0600` inside a private `0700` directory. With the +system unit above, they are owned by the `sdsctl` account. Run administrative +client checks as that account unless an intentionally designed local access +policy changes ownership or permissions outside the application. + +```bash +sudo -u sdsctl /opt/sdsctl/bin/sdsctl \ + daemon-client \ + --socket-path /run/sdsctl/daemon.sock \ + status + +sudo -u sdsctl /opt/sdsctl/bin/sdsctl \ + daemon-client \ + --socket-path /run/sdsctl/daemon.sock \ + events \ + --event-socket-path /run/sdsctl/events.sock \ + --count 10 + +sudo -u sdsctl /opt/sdsctl/bin/sdsctl \ + daemon-client \ + --socket-path /run/sdsctl/daemon.sock \ + audio \ + --pcmu-socket-path /run/sdsctl/pcmu.sock \ + --duration 10 \ + --output /var/lib/sdsctl/recordings/client-check.wav +``` + +A daemon-backed TUI needs all three explicit socket paths when it runs under a +different environment: + +```bash +sudo -u sdsctl /opt/sdsctl/bin/sdsctl tui \ + --daemon-client \ + --daemon-socket-path /run/sdsctl/daemon.sock \ + --daemon-event-socket-path /run/sdsctl/events.sock \ + --daemon-pcmu-socket-path /run/sdsctl/pcmu.sock +``` + +Closing a daemon-backed CLI or TUI client does not stop daemon ownership. + +## Upgrade from v0.18.x + +1. Record the installed version and service state. +2. Stop the service. +3. Back up `/etc/sdsctl/`, the service account's `sdsctl` and legacy `sds200` + configuration directories, and any recording or metadata paths. +4. Upgrade the package in the dedicated virtual environment. +5. Confirm both the CLI and Python package report the intended version. +6. Review the destination manifest and environment-backed secret references. +7. Start the service and verify API, event, PCMU, scanner, PSI, and audio health. +8. Exercise one daemon CLI client and one daemon-backed TUI client before + returning the service to normal operation. + +Example: + +```bash +sudo systemctl stop sdsctl.service + +sudo /opt/sdsctl/bin/python -m pip install --upgrade "sds200==0.19.0" + +/opt/sdsctl/bin/sdsctl --version +/opt/sdsctl/bin/python -c \ + "import sds200; print(sds200.__version__)" + +sudo systemctl start sdsctl.service +sudo systemctl status sdsctl.service +``` + +The upgrade does not rename the distribution, import package, executable, or +legacy profile stores. It does not move or rewrite configuration automatically. + +For rollback, stop the service, restore the previously installed package and +backed-up configuration, then restart and repeat the same health checks. Do not +move or reuse a published version tag. + +## Security and operational limits + +The SDS200 network protocols are unauthenticated and unencrypted. Keep scanner +control and RTSP/RTP audio on a trusted LAN or secured VPN. The daemon sockets +are local Unix-domain sockets and are not TCP services. + +The daemon does not fork, create a pidfile, change privileges, install a unit, +perform socket activation, or expose unrestricted raw scanner commands. +Decoded-PCM subscriptions and automatic daemon discovery remain follow-on work. diff --git a/docs/daemon-events.md b/docs/daemon-events.md index 7a6aeb0..abc5496 100644 --- a/docs/daemon-events.md +++ b/docs/daemon-events.md @@ -285,7 +285,7 @@ Validated on 2026-08-05 against a physical SDS200 at `192.168.0.251`: - the daemon returned exit status 0; and - both `daemon.sock` and `events.sock` were removed before process exit. -The initial daemon router contains no activated destinations, so -`destination.health` was not exercised by this hardware run. Destination -aggregation, overflow, unsubscribe, size, concurrency, and failure-isolation -behavior remain covered by hardware-independent regression tests. +Daemon-owned playback, recording, and remote-profile destinations now +participate in the shared router. Active destination lifecycle and health +changes are published through the existing `destination.health` event contract. +An empty destination set legitimately produces no destination-health events. diff --git a/docs/daemon-runtime.md b/docs/daemon-runtime.md index 80bdad8..6faa699 100644 --- a/docs/daemon-runtime.md +++ b/docs/daemon-runtime.md @@ -35,8 +35,9 @@ safe-control, ordered event-watch, and PCMU playback or WAV-recording workflows while preserving the standalone top-level scanner and direct-audio commands. Milestone 19.10 adds explicit daemon-backed TUI operation using the API, event, and PCMU services while preserving standalone TUI ownership as the default. -Decoded-PCM CLI subscriptions, automatic daemon discovery and selection, and -destination activation remain follow-on work. The process does not fork or +Milestone 19.11 adds validated saved-destination activation and transactional +`SIGHUP` replacement. Decoded-PCM CLI subscriptions and automatic daemon +discovery and selection remain follow-on work. The process does not fork or create a pidfile. ## Foreground process contract @@ -56,12 +57,13 @@ sdsctl --log-level INFO --profile home daemon The command constructs exactly one `DaemonRuntime`, one `PcmSinkRouter`, one `NetworkAudioTransport`, one compatibility-named `DaemonReadOnlyApi`, one bounded `DaemonApiServer`, one `DaemonEventStream`, one bounded -`DaemonEventServer`, one `PcmuStream`, and one bounded `DaemonPcmuServer`. The -API class retains its historical public name while exposing backward-compatible +`DaemonEventServer`, one `PcmuStream`, one bounded `DaemonPcmuServer`, one +`DaemonDestinationCoordinator`, and one `DaemonDestinationReloader`. The API +class retains its historical public name while exposing backward-compatible reads and explicit safe controls. The PCMU stream subscribes to the same -authoritative transport used by the decoded-PCM fanout. The router begins without -destinations because daemon-client audio consumes PCMU on the client side. -Daemon-owned destination activation, remote-profile activation, decoded-PCM +authoritative transport used by the decoded-PCM fanout. The coordinator activates +the validated startup destination set against the shared decoded-PCM router. +Daemon-client audio continues to consume PCMU independently. Decoded-PCM client subscriptions remain follow-on work. The audio endpoint must come from either `--host` or a network-capable SDS200 @@ -373,9 +375,16 @@ The Milestone 19.7 `pcmu.sock` service was physically validated on - controlled `SIGTERM` returned exit status 0 and removed `daemon.sock`, `events.sock`, and `pcmu.sock`. -The initial daemon router has no activated destinations, so -`destination.health` was not hardware-exercised. Its aggregation and isolation -contracts remain covered by hardware-independent regression tests. +Milestone 19.11 destination activation and reload were physically validated on +2026-08-06 against the same SDS200: + +- startup activated an initial recording destination; +- `SIGHUP` transactionally replaced it with recording plus audible playback; +- an invalid version 2 manifest failed with `ConfigurationError` while the + committed destinations and daemon runtime continued; +- a valid empty manifest removed all active destinations; +- finalized recordings remained valid 8 kHz mono signed 16-bit WAV files; and +- controlled `SIGTERM` returned exit status 0 and removed all three sockets. Milestone 19.8 safe-control contracts are covered by hardware-independent tests, including acknowledgements, rejection, deadlines, unsupported transports, @@ -423,12 +432,16 @@ running. Controlled `SIGTERM` subsequently removed all three sockets. ## Follow-on work -Later Milestone 19 work may: +Later work may: - add bounded decoded-PCM subscriptions for local clients; - add daemon discovery and automatic client selection; and - add decoded-PCM CLI client workflows. Decoded-PCM subscription, discovery, and automatic selection remain follow-on -work. Milestone 19.11 owns saved destination activation and validated `SIGHUP` -replacement. +work. Saved destination activation and validated `SIGHUP` replacement are part +of the current daemon contract. + +See [Daemon deployment and upgrade guide](daemon-deployment.md) for service +installation, explicit socket paths, destination manifests, reload, migration, +and upgrade procedures. diff --git a/docs/logging.md b/docs/logging.md index 04f15e8..62522de 100644 --- a/docs/logging.md +++ b/docs/logging.md @@ -74,10 +74,10 @@ delay. ## systemd and journald `sdsctl daemon` remains in the foreground so systemd can own process creation, -restart policy, termination, and log collection directly. Use `Type=simple`; -`Type=forking` and pidfiles are not part of the daemon contract. +restart policy, termination, reload, and log collection directly. Use +`Type=simple`; `Type=forking` and pidfiles are not part of the daemon contract. -A minimal unit using an explicit SDS200 network host is: +A minimal unit using explicit service paths is: ```ini [Unit] @@ -89,22 +89,25 @@ Wants=network-online.target Type=simple User=sdsctl Group=sdsctl -ExecStart=/opt/sdsctl/bin/sdsctl --log-level INFO --host 192.168.0.251 daemon +EnvironmentFile=-/etc/sdsctl/sdsctl.env +ExecStart=/opt/sdsctl/bin/sdsctl --log-level INFO --host 192.168.0.251 daemon --destination-config /etc/sdsctl/daemon-destinations.toml --socket-path /run/sdsctl/daemon.sock --event-socket-path /run/sdsctl/events.sock --pcmu-socket-path /run/sdsctl/pcmu.sock +ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure RestartSec=5 KillSignal=SIGTERM TimeoutStopSec=20 +RuntimeDirectory=sdsctl +RuntimeDirectoryMode=0700 +StateDirectory=sdsctl +StateDirectoryMode=0700 +UMask=0077 [Install] WantedBy=multi-user.target ``` Global connection and logging options precede the `daemon` subcommand. A saved -network-capable SDS200 profile may be used instead: - -```ini -ExecStart=/opt/sdsctl/bin/sdsctl --log-level INFO --profile home daemon -``` +network-capable SDS200 profile may replace the explicit host. No `--log-file` option is required for journald. Standard error is captured by the service manager. `SIGTERM` requests orderly runtime shutdown and normally @@ -112,10 +115,11 @@ returns success, so `Restart=on-failure` does not restart an intentionally stopped service. Startup or shutdown errors use a nonzero exit status and are eligible for restart. -`SIGHUP` requests a validated reload of the daemon destination manifest -without stopping the scanner, audio runtime, or local services. Use `SIGTERM`, -not `SIGHUP`, for controlled service termination. A failed reload leaves the -previous committed destinations active and is reported through the service log. +`SIGHUP` requests a validated reload of the exact daemon destination manifest +selected at startup without stopping scanner, audio, or local services. A failed +reload leaves the previous committed destinations active and is reported through +the service log. `systemctl reload sdsctl.service` uses `ExecReload` to send that +signal. Inspect the service log with: @@ -125,8 +129,9 @@ journalctl -u sdsctl.service --since today journalctl -u sdsctl.service -f ``` -Service installation, account creation, privilege changes, hardening directives, -and distribution-specific packaging remain administrator responsibilities. +Service account creation, installation, explicit socket access, destination +configuration, hardening, migration, upgrades, and rollback are covered by the +[daemon deployment and upgrade guide](daemon-deployment.md). ## `/var/log/sdsctl.log` diff --git a/scripts/check_docs.py b/scripts/check_docs.py index dab97da..65b3982 100644 --- a/scripts/check_docs.py +++ b/scripts/check_docs.py @@ -17,6 +17,7 @@ Path("SECURITY.md"), Path("SUPPORT.md"), Path("docs/configuration.md"), + Path("docs/daemon-deployment.md"), Path("docs/releasing.md"), Path("docs/supported-models.md"), Path("wiki/Home.md"), diff --git a/wiki/Home.md b/wiki/Home.md index 9288c6f..3319569 100644 --- a/wiki/Home.md +++ b/wiki/Home.md @@ -25,6 +25,8 @@ The project is not affiliated with or endorsed by Uniden. — full-screen monitoring, controls, recording, and playback. - [Network audio guide](https://github.com/stevenboyd78/sds200-python/blob/main/docs/audio.md) — SDS200 playback, recording, Broadcastify, and Asterisk integration. +- [Daemon deployment and upgrade guide](https://github.com/stevenboyd78/sds200-python/blob/main/docs/daemon-deployment.md) + — systemd, destination manifests, local clients, migration, and upgrades. ## Supported scanners @@ -82,6 +84,7 @@ sdsctl --host SCANNER_IP audio \ - [Roadmap](https://github.com/stevenboyd78/sds200-python/blob/main/ROADMAP.md) - [Changelog](https://github.com/stevenboyd78/sds200-python/blob/main/CHANGELOG.md) - [Operational logging](https://github.com/stevenboyd78/sds200-python/blob/main/docs/logging.md) +- [Daemon deployment and upgrades](https://github.com/stevenboyd78/sds200-python/blob/main/docs/daemon-deployment.md) - [Layered application configuration](https://github.com/stevenboyd78/sds200-python/blob/main/docs/configuration.md) - [Capture and replay](https://github.com/stevenboyd78/sds200-python/blob/main/docs/replay-and-capture.md) - [Linux udev rule](https://github.com/stevenboyd78/sds200-python/blob/main/docs/udev.md) diff --git a/wiki/Installation.md b/wiki/Installation.md index 48fe893..c66b497 100644 --- a/wiki/Installation.md +++ b/wiki/Installation.md @@ -132,6 +132,38 @@ sdsctl --host SCANNER_IP info Only scan networks you own or are authorized to probe. +## Upgrade from v0.18.x + +The v0.19.0 release keeps the distribution and Python import package named +`sds200` and keeps the executable named `sdsctl`. Application and service paths +use the `sdsctl` namespace, while existing scanner and remote-audio profiles +remain under the legacy `sds200` configuration root. + +No file is moved or rewritten automatically. Before upgrading, back up system +and user configuration, legacy profile files, destination manifests, recordings, +and metadata. Upgrade the package, verify `sdsctl --version` and +`sds200.__version__`, then exercise both standalone and daemon-backed workflows. + +See the canonical +[daemon deployment and upgrade guide](https://github.com/stevenboyd78/sds200-python/blob/main/docs/daemon-deployment.md) +for a complete systemd unit, destination manifest, reload, client, upgrade, and +rollback procedure. + +## Run the SDS200 daemon + +The foreground daemon is intended for process-manager ownership. It exposes +private local API, event, and PCMU sockets and can activate saved playback, +recording, and remote-profile destinations. + +```bash +sdsctl --log-level INFO --host SCANNER_IP daemon +sdsctl daemon-client status +sdsctl tui --daemon-client +``` + +Standalone scanner commands and the standalone TUI remain the default. Daemon +client mode is explicit. + ## Next steps - Launch the terminal monitor with `sdsctl monitor`. diff --git a/wiki/_Sidebar.md b/wiki/_Sidebar.md index eb2b0fc..7f0faad 100644 --- a/wiki/_Sidebar.md +++ b/wiki/_Sidebar.md @@ -11,6 +11,7 @@ - [Textual TUI](https://github.com/stevenboyd78/sds200-python/blob/main/docs/tui.md) - [Network audio](https://github.com/stevenboyd78/sds200-python/blob/main/docs/audio.md) - [Operational logging](https://github.com/stevenboyd78/sds200-python/blob/main/docs/logging.md) +- [Daemon deployment](https://github.com/stevenboyd78/sds200-python/blob/main/docs/daemon-deployment.md) - [Layered configuration](https://github.com/stevenboyd78/sds200-python/blob/main/docs/configuration.md) - [Capture and replay](https://github.com/stevenboyd78/sds200-python/blob/main/docs/replay-and-capture.md) From f7c2a9e93056f2eab188bd4ac53cb64788a07811 Mon Sep 17 00:00:00 2001 From: stevenboyd78 Date: Thu, 6 Aug 2026 03:52:51 -0600 Subject: [PATCH 2/5] Add CodeQL analysis workflow --- .github/workflows/codeql.yml | 43 ++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 .github/workflows/codeql.yml diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 0000000..1c364e4 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,43 @@ +name: CodeQL + +on: + push: + branches: + - main + pull_request: + branches: + - main + schedule: + - cron: "23 8 * * 1" + +permissions: + contents: read + +concurrency: + group: codeql-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + analyze: + name: Analyze Python + runs-on: ubuntu-latest + + permissions: + actions: read + contents: read + security-events: write + + steps: + - name: Check out repository + uses: actions/checkout@v7 + + - name: Initialize CodeQL + uses: github/codeql-action/init@v4 + with: + languages: python + build-mode: none + + - name: Analyze + uses: github/codeql-action/analyze@v4 + with: + category: /language:python From 06a00e76836582eca50b6eb11787dac2fcaeaa7d Mon Sep 17 00:00:00 2001 From: stevenboyd78 Date: Thu, 6 Aug 2026 04:03:43 -0600 Subject: [PATCH 3/5] Prepare v0.19.0 release metadata --- CHANGELOG.md | 12 +++++++++++- ROADMAP.md | 21 ++++++++++++--------- pyproject.toml | 2 +- src/sds200/__init__.py | 2 +- 4 files changed, 25 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5bd42bb..8fca6a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,8 @@ to follow [Semantic Versioning](https://semver.org/) as the public API matures. ## [Unreleased] +## [0.19.0] - 2026-08-06 + ### Added - Immutable renderer-neutral application configuration values with validated @@ -143,6 +145,13 @@ to follow [Semantic Versioning](https://semver.org/) as the public API matures. Quitting the TUI left scanner, PSI, RTSP/RTP audio, router, and daemon ownership running. Controlled `SIGTERM` then removed all three sockets. +- A daemon deployment and upgrade guide covering preserved naming + compatibility, dedicated service accounts, systemd supervision, private socket + access, destination manifests, transactional reload, migration, clean + installation, upgrades, and rollback. +- A GitHub CodeQL workflow that analyzes Python changes on main-branch pushes, + pull requests, and a weekly schedule. + ### Changed - `sdsctl` now resolves application settings from built-in defaults, @@ -701,7 +710,8 @@ First planned GitHub prerelease. - Added serial discovery, transport, packet framing, core responses, CLI tools, examples, tests, and CI. -[Unreleased]: https://github.com/stevenboyd78/sds200-python/compare/v0.18.0...HEAD +[Unreleased]: https://github.com/stevenboyd78/sds200-python/compare/v0.19.0...HEAD +[0.19.0]: https://github.com/stevenboyd78/sds200-python/compare/v0.18.0...v0.19.0 [0.18.0]: https://github.com/stevenboyd78/sds200-python/compare/v0.17.0...v0.18.0 [0.17.0]: https://github.com/stevenboyd78/sds200-python/compare/v0.16.1...v0.17.0 [0.16.1]: https://github.com/stevenboyd78/sds200-python/compare/v0.16.0...v0.16.1 diff --git a/ROADMAP.md b/ROADMAP.md index 31c9c09..28119d8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -13,16 +13,19 @@ and ideas that are not ready for scheduling are recorded in ### Milestone 19.12 — v0.19.0 release -- **Compatibility, migration, deployment, and systemd documentation — active** - - Document preserved distribution, import-package, executable, and legacy +- **Compatibility, migration, deployment, and systemd documentation — complete** + - Documented preserved distribution, import-package, executable, and legacy profile compatibility. - - Document explicit destination manifests, service accounts, private sockets, - `SIGHUP` reload, systemd operation, upgrades, and rollback. -- **Acceptance validation — pending** - - Validate multiple clients, slow and malformed clients, shutdown fault - injection, clean installation, and upgrade behavior. - - Run the full static, test, documentation, packaging, and distribution suite. - - Complete physical SDS200 daemon-owned CLI and TUI client validation. + - Documented explicit destination manifests, service accounts, private + sockets, `SIGHUP` reload, systemd operation, upgrades, and rollback. +- **Acceptance validation — active** + - Completed adversarial regression validation for multiple, slow, malformed, + and excess clients plus shutdown fault injection. + - Completed preliminary source and wheel builds, Twine checks, artifact + inspection, clean installation, and public-to-local upgrade validation. + - Added Python CodeQL analysis; repeat package validation against `0.19.0`, + run full Python 3.11–3.14 CI and CodeQL, and complete final physical SDS200 + daemon-owned CLI and TUI client validation. - **Publication — pending** - Merge release preparation only after CI and CodeQL pass. - Publish `v0.19.0` through trusted PyPI publishing and create the GitHub diff --git a/pyproject.toml b/pyproject.toml index 58f5b5a..28418f2 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "sds200" -version = "0.18.0" +version = "0.19.0" description = "Python control library for Uniden SDS100, SDS150, and SDS200 scanners" readme = "README.md" requires-python = ">=3.11" diff --git a/src/sds200/__init__.py b/src/sds200/__init__.py index 57c47f7..c94db26 100644 --- a/src/sds200/__init__.py +++ b/src/sds200/__init__.py @@ -893,4 +893,4 @@ "write_recording_metadata", ] -__version__ = "0.18.0" +__version__ = "0.19.0" From a359e90dd3d7668abc90bcef74a27957c90decc1 Mon Sep 17 00:00:00 2001 From: stevenboyd78 Date: Thu, 6 Aug 2026 05:44:37 -0600 Subject: [PATCH 4/5] Use repository CodeQL default setup --- .github/workflows/codeql.yml | 43 ------------------------------------ CHANGELOG.md | 4 ++-- ROADMAP.md | 7 +++--- 3 files changed, 6 insertions(+), 48 deletions(-) delete mode 100644 .github/workflows/codeql.yml diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml deleted file mode 100644 index 1c364e4..0000000 --- a/.github/workflows/codeql.yml +++ /dev/null @@ -1,43 +0,0 @@ -name: CodeQL - -on: - push: - branches: - - main - pull_request: - branches: - - main - schedule: - - cron: "23 8 * * 1" - -permissions: - contents: read - -concurrency: - group: codeql-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -jobs: - analyze: - name: Analyze Python - runs-on: ubuntu-latest - - permissions: - actions: read - contents: read - security-events: write - - steps: - - name: Check out repository - uses: actions/checkout@v7 - - - name: Initialize CodeQL - uses: github/codeql-action/init@v4 - with: - languages: python - build-mode: none - - - name: Analyze - uses: github/codeql-action/analyze@v4 - with: - category: /language:python diff --git a/CHANGELOG.md b/CHANGELOG.md index 8fca6a0..3ba6292 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -149,8 +149,8 @@ to follow [Semantic Versioning](https://semver.org/) as the public API matures. compatibility, dedicated service accounts, systemd supervision, private socket access, destination manifests, transactional reload, migration, clean installation, upgrades, and rollback. -- A GitHub CodeQL workflow that analyzes Python changes on main-branch pushes, - pull requests, and a weekly schedule. +- Validated the repository's existing GitHub CodeQL default setup for Actions + and Python against the release pull-request head with no analysis errors. ### Changed diff --git a/ROADMAP.md b/ROADMAP.md index 28119d8..3a60c0b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -23,9 +23,10 @@ and ideas that are not ready for scheduling are recorded in and excess clients plus shutdown fault injection. - Completed preliminary source and wheel builds, Twine checks, artifact inspection, clean installation, and public-to-local upgrade validation. - - Added Python CodeQL analysis; repeat package validation against `0.19.0`, - run full Python 3.11–3.14 CI and CodeQL, and complete final physical SDS200 - daemon-owned CLI and TUI client validation. + - Validated the repository's existing GitHub CodeQL default setup for Actions + and Python; repeat package validation against `0.19.0`, run the full Python + 3.11–3.14 CI matrix, and complete final physical SDS200 daemon-owned CLI and + TUI client validation. - **Publication — pending** - Merge release preparation only after CI and CodeQL pass. - Publish `v0.19.0` through trusted PyPI publishing and create the GitHub From 5b02f0bd7fcafb88388eab49361e92652754a434 Mon Sep 17 00:00:00 2001 From: stevenboyd78 Date: Thu, 6 Aug 2026 05:53:02 -0600 Subject: [PATCH 5/5] Record v0.19.0 acceptance completion --- ROADMAP.md | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 3a60c0b..e0568dd 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -18,15 +18,21 @@ and ideas that are not ready for scheduling are recorded in profile compatibility. - Documented explicit destination manifests, service accounts, private sockets, `SIGHUP` reload, systemd operation, upgrades, and rollback. -- **Acceptance validation — active** - - Completed adversarial regression validation for multiple, slow, malformed, - and excess clients plus shutdown fault injection. - - Completed preliminary source and wheel builds, Twine checks, artifact - inspection, clean installation, and public-to-local upgrade validation. - - Validated the repository's existing GitHub CodeQL default setup for Actions - and Python; repeat package validation against `0.19.0`, run the full Python - 3.11–3.14 CI matrix, and complete final physical SDS200 daemon-owned CLI and - TUI client validation. +- **Acceptance validation — complete** + - Passed adversarial regression validation for multiple, slow, malformed, and + excess clients plus shutdown fault injection. + - Passed final Ruff, MyPy for 74 source files, all 1,267 tests on Python + 3.14.4, documentation checks for 34 Markdown files, version `0.19.0` source + and wheel builds, Twine checks, artifact inspection, clean installation, and + public PyPI `0.18.0` to local `0.19.0` upgrade validation. + - Passed the full Python 3.11–3.14 GitHub Actions matrix and the repository's + existing CodeQL default setup for Actions and Python with zero findings and + no analysis errors. + - Completed physical SDS200 daemon-owned CLI and TUI validation against + firmware `1.26.01`, including private sockets, simultaneous clients, + canonical `stream.snapshot` delivery, loss-free PCMU reception, valid WAV + and metadata output, daemon-owned reconnect, retained ownership after TUI + exit, and orderly shutdown with all sockets removed. - **Publication — pending** - Merge release preparation only after CI and CodeQL pass. - Publish `v0.19.0` through trusted PyPI publishing and create the GitHub