Skip to content

Implement the visualizer@v1 role (aiosendspin 6.x compatibility) - #36

Open
ecohash-co wants to merge 2 commits into
Sendspin:mainfrom
ecohash-co:upstream-6x-conformance
Open

Implement the visualizer@v1 role (aiosendspin 6.x compatibility)#36
ecohash-co wants to merge 2 commits into
Sendspin:mainfrom
ecohash-co:upstream-6x-conformance

Conversation

@ecohash-co

Copy link
Copy Markdown

Summary

The visualizer role is currently a placeholder: client/hello encodes
visualizer@v1_support as an empty {}, the stream/start visualizer block decodes
to an empty struct, and only binary type 16 is recognized. Against aiosendspin 6.x
(Music Assistant 2.9.x) this is worse than "not implemented": the server validates
the hello support object strictly and hard-rejects the entire client/hello when
visualizer@v1 is advertised with an empty support object
, so turning the role on
breaks the whole connection during the handshake. (aiosendspin
models/visualizer.py ClientHelloVisualizerSupport requires non-empty types,
positive buffer_capacity and rate_max, and a spectrum object whenever
"spectrum" is in types.)

This PR implements the role end-to-end, following the existing artwork-role patterns:

  • VisualizerConfiguration (public, throws(ConfigurationError), mirrors
    ArtworkConfiguration): feature types (loudness / beat / f_peak / spectrum /
    peak / pitch), spectrum binning, bufferCapacity, rateMax. SendspinClient
    gains a visualizerConfig: parameter; advertising .visualizerV1 without one
    throws .visualizerRoleRequiresConfiguration, so the rejected empty shape can no
    longer be sent (same pattern as .playerRoleRequiresConfiguration).
  • client/hello: the internal VisualizerSupport wire struct now carries the
    validated real fields (buffer_capacity, rate_max, types, spectrum).
  • stream/start: the negotiated visualizer announcement decodes into the public
    StreamVisualizerConfig (types, rate_max, tracks_downbeats, spectrum) and
    surfaces as ClientEvent.visualizerStreamStarted(_:) +
    SendspinClient.currentVisualizerStream. Decoding is tolerant: unknown feature
    strings are dropped and a malformed visualizer block cannot fail the whole
    stream/start (which would also kill the player section and silence audio).
  • Binary types 17–21 (beat, f_peak, spectrum, peak, pitch — aiosendspin
    models/types.py BinaryMessageType) are recognized and routed through the
    existing gate/clock-sync checks to the visualizerData stream; previously they
    were dropped as unknown type IDs at BinaryMessage.init. Type 16
    (.visualizerData) is renamed .visualizerLoudness per the per-feature ID
    allocation (internal enum, no public API break).
  • VisualizerData now carries the feature type (without it, consumers of the
    multi-type stream can't tell a loudness frame from a beat frame — the type byte is
    stripped with the header) and offers frame(spectrumBins:) decoding into the new
    typed VisualizerFrame enum. Payload layouts follow the reference wire packing
    (server/roles/visualizer/v1.py, all big-endian: loudness u16; beat flags byte,
    bit 0 = downbeat; f_peak u16 freq + u16 amp; spectrum n_disp_bins × u16; peak
    u8; pitch u16 MIDI Q8.8 + u8 confidence). Wrong-length payloads reject rather
    than mis-decode; spectrum frames require the negotiated bin count for sizing.

One field-derived default worth calling out: rateMax defaults to 30 Hz, not
60.
aiosendspin queues visualizer frames per-role for the entire audio write-ahead
window in a 4096-slot queue; at 5 feature types × 60 Hz (300 frames/s), the queue
overflows once write-ahead exceeds ~13.6 s — which a 2 MB player buffer_capacity
reaches on well-compressed FLAC — and the server then disconnects the client
("Role queue full for visualizer"). 30 Hz keeps a 5-type client safe past 27 s of
write-ahead. Found and verified against a live Music Assistant 2.9.8 /
aiosendspin 6.0.5 server.

Commits

  1. feat(visualizer): implement the visualizer@v1 role per aiosendspin 6.0.5
    models + client wiring + CHANGELOG, plus mechanical updates of the three
    existing tests that referenced .visualizerData / the empty
    StreamStartVisualizer.
  2. test(visualizer): aiosendspin 6.0.5 conformance fixtures — 15 golden-message
    tests hand-derived from the reference implementation
    (Tests/SendspinKitTests/Models/VisualizerModelTests.swift), including the
    regression guard that the encoded support object can never again be {}.

Test evidence

On macOS (Apple Silicon, Swift 6.2.3 / Xcode 26.2 toolchain):

  • swift test567 tests in 50 suites, all passing (upstream main is 552;
    +15 new visualizer conformance tests; all pre-existing tests green, including the
    updated FrameOrderingTests visualizer gate/clock-sync ordering tests).
  • swiftlint lint --strict — clean.
  • swiftformat --lint (0.62.1) — all files touched by this PR are clean. (Note:
    0.62.1 flags ~15 untouched files on current main — e.g.
    GroupUpdatePayload.encode's single-line if bodies — a rule-drift of the newer
    swiftformat vs. the version CI installed on the last main run. Not introduced
    here.)

Compatibility

  • No breaking public API changes: visualizerConfig: is a new defaulted init
    parameter; VisualizerData gains a field (it has no public initializer);
    StreamRole gains .visualizer; new public types are additive.
  • Wire behavior change: clients that previously advertised .visualizerV1 (and were
    rejected by 6.x servers at hello) now fail fast at SendspinClient.init unless
    they provide a configuration.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CbdWgqSEcL2JhwBkwabRy3

Jordan Miller and others added 2 commits July 18, 2026 22:42
The visualizer role was a placeholder: client/hello encoded
visualizer@v1_support as an empty {}, stream/start's visualizer block decoded
to an empty struct, and only binary type 16 was recognized. aiosendspin 6.x
(Music Assistant 2.9.x) validates the hello support object strictly —
non-empty types, positive buffer_capacity/rate_max, spectrum config when
"spectrum" is advertised — and hard-rejects the whole client/hello otherwise,
so advertising the role broke the handshake against a live server.

- client/hello: VisualizerSupport now carries the validated real fields
  (buffer_capacity, rate_max, types, spectrum). The support object is built
  from a new public VisualizerConfiguration (throws ConfigurationError on
  invalid input, matching the player/artwork config pattern), and creating a
  client that advertises visualizer@v1 without one throws
  .visualizerRoleRequiresConfiguration — the rejected empty shape can no
  longer be sent.
- stream/start: the negotiated visualizer announcement decodes into the
  public StreamVisualizerConfig (types / rate_max / tracks_downbeats /
  spectrum). Decoding is tolerant — unknown feature strings are dropped and a
  malformed block does not fail the whole stream/start (which would also
  silence the player section). Surfaced as
  ClientEvent.visualizerStreamStarted and currentVisualizerStream; the
  spectrum bin count there sizes binary spectrum frames.
- Binary types 17-21 (beat, f_peak, spectrum, peak, pitch) are recognized and
  routed to the visualizer data stream; previously those frames were dropped
  as unknown type IDs. .visualizerData (16) is renamed .visualizerLoudness to
  match aiosendspin's per-feature ID allocation. VisualizerData now carries
  the feature type and offers frame(spectrumBins:) decoding into the typed
  VisualizerFrame enum (wire layouts from server/roles/visualizer/v1.py, all
  big-endian).
- The default rate_max is a deliberately conservative 30 Hz: the server
  queues frames per-role for the whole audio write-ahead window, and 5 types
  x 60 Hz overflows aiosendspin's 4096-slot role queue once write-ahead
  exceeds ~13.6 s, after which the server disconnects the client ("Role
  queue full for visualizer"). Verified against a live Music Assistant
  2.9.8 / aiosendspin 6.0.5 server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbdWgqSEcL2JhwBkwabRy3
Golden-message fixtures hand-derived from the aiosendspin 6.0.5 reference
implementation (models/visualizer.py, models/types.py,
server/roles/visualizer/v1.py):

- client/hello visualizer@v1_support encodes every server-validated field and
  can never regress to the rejected empty {} shape; absent spectrum is an
  omitted key, not null.
- The facade builds the full support object from VisualizerConfiguration, and
  the role cannot be advertised without one.
- VisualizerConfiguration/VisualizerSpectrumConfig reject the same inputs the
  server rejects (empty types, non-positive capacity/rate/bins, missing
  spectrum config, inverted frequency range).
- stream/start visualizer negotiation decodes alongside the player section,
  and unknown feature strings are dropped without failing the message.
- Binary frames 16-21 decode per the reference wire packing (">Bq" header;
  loudness u16, beat flags bit 0, f_peak u16+u16, spectrum n_disp_bins x u16,
  peak u8, pitch u16 Q8.8 + confidence u8), and wrong-length payloads reject
  rather than mis-decode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbdWgqSEcL2JhwBkwabRy3

@teancom teancom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing! Thank you so much. One small nit and then it looks great.

)
case .spectrum:
// v1.py — spectrum.astype(">u2"), n_disp_bins values.
guard spectrumBins > 0, payload.count == spectrumBins * 2 else { return nil }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A malformed server could send spectrumBins = Int.max, and then we trap when we try to double it. Please use checked multiplication or validate it in some other way before doing this calculation. A test for this would be great as well.

@teancom

teancom commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Oh, also I just fixed all of the existing swiftlint/swiftformat errors. If you could rebase or update the branch, it would make CI pass. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants