Skip to content

internals.md

Latest commit

 

History

History
514 lines (363 loc) · 51.4 KB

internals.md

File metadata and controls

514 lines (363 loc) · 51.4 KB
Raw

Sendspin-cpp Internals

This document describes the internal architecture of the sendspin-cpp library, focusing on threading, inter-class communication, and the ordering guarantees that keep everything correct.

Pimpl Architecture

Each role class uses the pimpl (pointer to implementation) pattern. The public header (include/sendspin/<role>_role.h) exposes only the consumer-facing API: protocol types, the listener interface, and a thin role class with struct Impl; std::unique_ptr<Impl> impl_;. All private state, internal methods, and thread-management code live in a private impl header (src/<role>_role_impl.h) and the corresponding .cpp file.

SendspinClient is a friend of each role class, giving it access to impl_-> for internal dispatch (message routing, event draining, lifecycle management). The SyncTask holds a PlayerRole::Impl* directly (passed at init time), so it accesses player state without indirection through the public PlayerRole class.

Throughout this document, internal field and method references use the Impl qualification (e.g., PlayerRole::Impl::drain_events()) to reflect the actual code location.

Conditional Compilation

Roles can be disabled at build time via SENDSPIN_ENABLE_* (CMake options on host, Kconfig entries on ESP-IDF). Two mechanisms cooperate, with a strict boundary between them:

  1. CMake source-list exclusion (cmake/sources.cmake). Each role has its own SENDSPIN_<ROLE>_SOURCES list. When a role is disabled, its translation units are not added to the build, so the code never compiles and its transitive dependencies are not required; e.g., micro-flac and micro-opus for the player. The ESP-IDF component manifest (idf_component.yml) similarly gates the audio codec dependencies on SENDSPIN_ENABLE_PLAYER so they are not even fetched.
  2. #ifdef SENDSPIN_ENABLE_<ROLE> guards in include/sendspin/client.h and src/client.cpp. These are the only core files that must reference role types directly (the std::unique_ptr<RoleClass> members, add_*() / accessor declarations, and dispatch branches in message handlers). Nowhere else in the core should use these guards.

The split exists because the two problems are different. CMake handles "don't compile this file and don't require its dependencies," while #ifdef handles "core code needs to conditionally mention a type." Using #ifdef to gate entire files would still force the codec headers onto the include path; using CMake to gate individual member declarations is not possible.

As a consequence, role-only headers;e.g., src/decoder.h, which pulls in <micro_flac/flac_decoder.h> and <opus.h>, must only be reachable through role-only sources or through #ifdef-guarded includes in client.cpp. Public role headers in include/sendspin/ must remain free of codec dependencies so that core files like src/transfer_buffer.cpp and src/protocol_messages.h can include them unconditionally.

When adding a new role, the checklist is: add a SENDSPIN_<ROLE>_SOURCES list in cmake/sources.cmake, add the member/accessor/dispatch guards in client.h and client.cpp, and keep any heavy dependencies behind the role's private headers.

A separate #ifdef axis lives in src/platform/: headers there use #ifdef ESP_PLATFORM to select between ESP-IDF (FreeRTOS, heap_caps_malloc, esp_log, etc.) and host (std primitives, malloc, printf) implementations behind a common API. This is orthogonal to role selection; the split is between build targets, not features. Core sources outside src/platform/, src/esp/, and src/host/ should never use #ifdef ESP_PLATFORM directly, so platform differences stay isolated to the abstraction layer.

Thread Model

The library uses a small number of long-lived threads. All state mutations and user-facing callbacks happen on the caller's main loop thread unless explicitly noted otherwise.

Threads

Thread Name Created by Stack (ESP) Priority (ESP) Purpose
Main loop (caller's) User code - - Drives SendspinClient::loop(). All role event processing and listener callbacks run here.
Sync task Sendspin PlayerRole::Impl::start() → SyncTask::start() 6192 B 2 Decodes audio, synchronizes to server timestamps, writes PCM to the audio sink via on_audio_write.
Visualizer drain SsVis VisualizerRole::Impl::start() 4096 B 2 Reads visualization frames from a ring buffer and delivers them to the listener at the correct playback time.
Artwork decode SsArt ArtworkRole::Impl::start() 4096 B 2 Receives image notifications and calls the decode callback. Hands the server display timestamp off to the main loop, which fires the display callback at the correct time.
Network (library-internal) IXWebSocket (host) or esp_http_server (ESP) - - WebSocket I/O. Callbacks fire on these threads and must defer work to the main loop.

On host builds, platform_configure_thread() is a no-op; threads use OS defaults. On ESP-IDF it calls esp_pthread_set_cfg() to set stack size, priority, name, and optional PSRAM allocation before the std::thread is constructed.

Thread Lifecycle

Sync task (src/sync_task.cpp:620):

  1. SyncTask::start() configures the thread and spawns it.
  2. The caller blocks until the thread reaches IDLE state (TASK_IDLE event flag) or exits early due to an allocation failure (TASK_STOPPED).
  3. The thread runs a persistent outer loop for the lifetime of the client.
  4. SyncTask::stop() sets COMMAND_STOP and joins the thread. Called from SyncTask's destructor, which is triggered by sync_task_.reset() in PlayerRole::Impl's destructor.

Visualizer drain (src/visualizer_role.cpp):

  1. VisualizerRole::Impl::start() spawns the drain thread.
  2. The thread blocks on ring buffer receives with a 50 ms timeout.
  3. VisualizerRole::Impl destructor sets COMMAND_STOP and joins.

Artwork decode (src/artwork_role.cpp):

  1. ArtworkRole::Impl::start() spawns the decode thread.
  2. The thread blocks on notification queue receives with a 100 ms timeout.
  3. On notification: calls on_image_decode(), then merges an ArtworkDisplayUpdate (the slot's server display timestamp plus the stream_epoch it was decoded under) into the ArtworkRole::Impl::EventState::display_slot InboxSlot via merge_artwork_display_update. The main loop's ArtworkRole::Impl::drain_events() folds the taken update into its main-thread-only held_display_* state and fires on_image_display() once the timestamp is reached. Latest-wins per slot: if a newer frame's timestamp overwrites the pending one before the main loop takes it, only the newer display fires; the per-slot epoch lets the deadline sweep drop a display whose stream was replaced after the hand-off.
  4. ArtworkRole::Impl destructor sets COMMAND_STOP and joins.

Destruction order matters because external audio callbacks may still reference the sync task. PlayerRole::Impl's destructor resets the sync task first (sync_task_.reset()) before tearing down anything else, so the thread is fully joined before any shared state is destroyed.

Synchronization Primitives

All primitives are abstracted in src/platform/ with ESP-IDF (FreeRTOS) and host (std::mutex/condition_variable) implementations.

EventFlags (src/platform/event_flags.h)

Atomic bit flags with blocking wait. Used for thread lifecycle control:

COMMAND_STOP         (1 << 0)   Stop the thread
COMMAND_STREAM_END   (1 << 1)   End current stream
COMMAND_STREAM_CLEAR (1 << 2)   Seek: discard buffered audio up to the stream/clear marker
COMMAND_START        (1 << 3)   Main loop acknowledged stream start
TASK_RUNNING         (1 << 8)   Actively decoding
TASK_STOPPED         (1 << 10)  Thread has exited
TASK_ERROR           (1 << 11)  Allocation or decode failure
TASK_IDLE            (1 << 12)  Waiting for work

The sync task, visualizer drain thread, and artwork decode thread all use event flags for command signaling from the main loop and status reporting back. The artwork decode thread uses a simpler subset: just COMMAND_STOP. The visualizer drain thread adds COMMAND_FLUSH and COMMAND_CLEAR. COMMAND_CLEAR discards buffered entries up to a 1-byte marker the network thread enqueues on stream/start and stream/clear (mirroring the sync task's clear-marker chunk) so frames received after the boundary survive; COMMAND_FLUSH (drain to empty) is only used when the producer is already stopped (stream/end, cleanup).

ThreadSafeQueue (src/platform/thread_safe_queue.h)

Fixed-depth FIFO queue with timed send/receive. Used to defer events from network threads to the main loop:

Queue Depth Data Producer Consumer
ArtworkRole::Impl::notify_queue 8 ArtworkNotification Network thread Artwork decode thread

ShadowSlot (src/platform/shadow_slot.h)

Single-slot state container with "latest wins" or custom merge semantics for a producer/consumer thread pair where neither thread is the main loop. The writer thread writes or merges; the reader thread takes the accumulated value. Main-loop-bound cross-thread state goes through the Inbox / InboxSlot instead (see below), which consolidates many producers onto one shared mutex and a single lock-free poll() read per tick. Its one remaining user is the sync task's playback-progress slot (audio-callback thread → sync-task thread):

Shadow Slot Data Merge Strategy
SyncTask::playback_progress_slot_ PlaybackProgress Sum frames_played, keep latest finish_timestamp

The field-by-field merge pattern - preserving, say, a volume change and a mute change that arrive between two drain ticks because the merge only overwrites fields present in the delta - now lives on the Inbox merge slots (the player's command_slot, and the metadata/color/group delta slots below) rather than on a ShadowSlot.

Inbox (src/inbox.h)

Shared main-loop mailbox that consolidates client-level cross-thread traffic behind a single mutex and a lock-free dirty-topic bitmask. It provides two endpoint styles:

  • A fixed-capacity event ring (push_event / take_events) for lifecycle events, discriminated by InboxEventType. The ring drops on full (matching the old queue behavior) and the producer logs the drop.
  • InboxSlot<T>, a latest-value slot (write / merge / take) that replaces ShadowSlot for a single topic. Each slot exclusively owns one INBOX_TOPIC_* bit.

The main loop calls poll() once per tick to read the bitmask lock-free and only locks the mutex to drain topics whose bit is set. A bit set by a producer just after the snapshot is never lost: it stays set until a consumer drains it, so the next tick observes it. The Inbox is a leaf in the lock order, and merge functors must be pure data operations (no callbacks into application code while the mutex is held).

State currently on the Inbox:

Endpoint Topic bit Data Producer
Event ring INBOX_TOPIC_EVENTS Lifecycle events (TimeResponsePayload, PLAYER_STREAM STREAM_START/STREAM_END, ARTWORK_STREAM STREAM_END/STREAM_CLEAR, VISUALIZER_STREAM STREAM_START/STREAM_END/STREAM_CLEAR, plus CONTROLLER_CLEARED / METADATA_CLEARED / COLOR_CLEARED) via InboxEvent Network thread (TimeResponsePayload, PLAYER_STREAM, ARTWORK_STREAM, VISUALIZER_STREAM) / main-loop thread (*_CLEARED and the synthetic cleanup() stream events)
Client::group_slot INBOX_TOPIC_GROUP GroupUpdateObject (field-by-field delta merge) Network thread
ControllerRole::Impl::slot INBOX_TOPIC_CONTROLLER ServerStateControllerObject (latest wins) Network thread
MetadataRole::Impl::slot INBOX_TOPIC_METADATA ServerMetadataStateDelta (field-by-field delta merge) Network thread
ColorRole::Impl::slot INBOX_TOPIC_COLOR ServerColorStateDelta (field-by-field delta merge) Network thread
PlayerRole::Impl::EventState::stream_params_slot INBOX_TOPIC_PLAYER_STREAM_PARAMS ServerPlayerStreamObject (latest wins) Network thread
PlayerRole::Impl::EventState::command_slot INBOX_TOPIC_PLAYER_COMMAND ServerCommandMessage (field-by-field merge) Network thread
PlayerRole::Impl::EventState::state_slot INBOX_TOPIC_PLAYER_STATE SendspinClientState (latest wins) Sync task thread
VisualizerRole::Impl::EventState::config_slot INBOX_TOPIC_VISUALIZER_CONFIG ServerVisualizerStreamObject (latest wins) Network thread
ArtworkRole::Impl::EventState::display_slot INBOX_TOPIC_ARTWORK_DISPLAY ArtworkDisplayUpdate (per-slot display timestamp + epoch, merged) Artwork decode thread

All roles have been migrated onto the Inbox. The controller/metadata/color roles write or merge server state into their InboxSlot from handle_server_state(), and their disconnect clear arrives as a *_CLEARED lifecycle event on the shared ring rather than a per-role flag. The player role owns three InboxSlots (stream params, command, and client state, on its EventState) plus PLAYER_STREAM lifecycle events on the shared ring; its disconnect clear is the synthetic STREAM_END that cleanup() pushes onto the ring. The visualizer role writes its stream config to config_slot and delivers STREAM_START/END/CLEAR as VISUALIZER_STREAM ring events (no per-tick drain_events(); the config is taken when the START event is dispatched). The artwork role merges per-slot display timestamps (tagged with the decode stream_epoch) into display_slot, delivers STREAM_END/CLEAR as ARTWORK_STREAM ring events, and keeps a per-tick drain_events() for its server-clock display-deadline sweep. Both roles' disconnect clears are synthetic stream events that cleanup() pushes onto the ring.

SpscRingBuffer (src/platform/spsc_ring_buffer.h)

Single-producer/single-consumer ring buffer for variable-size binary data. Two-phase API: acquire → commit (producer), receive → return_item (consumer). Also supports a single-phase send for the producer.

Used for:

  • Encoded audio: Via the SendspinAudioRingBuffer wrapper (which adds chunk headers and exposes write_chunk / receive_chunk / return_chunk). Network thread writes chunks; sync task reads and decodes them.
  • Visualizer frames: Used directly. Network thread writes one entry per visualizer binary message; drain thread reads them at the correct playback time.

Other Primitives

  • std::mutex on ConnectionManager::conn_mutex_: protects deferred connection event vectors.
  • std::mutex on SendspinTimeFilter::state_mutex_: protects Kalman filter state (offset, drift, covariance).
  • std::atomic<bool> on SendspinConnection::message_dispatch_enabled_: allows the main loop to instantly suppress message delivery from the network thread.
  • std::atomic<bool/uint8_t> on VisualizerRole::Impl: network thread writes stream config atomically; drain thread reads it.
  • std::atomic<bool> on ArtworkRole::Impl::stream_active: guards handle_binary() from writing when no stream is active.
  • std::atomic<uint8_t> on ArtworkRole::Impl::SlotBuffer::write_idx: tracks which of two double-buffers the network thread writes to next.
  • std::atomic<bool> on ArtworkRole::Impl::SlotBuffer::drain_active: set by the decode thread while decoding, checked by the network thread to avoid overwriting an in-use buffer.
  • std::atomic<uint8_t> on SendspinClient::high_performance_ref_count_: ref-counted high-performance networking requests from time sync and playback.

Message Flow

WebSocket Receive Path

Network thread (IXWebSocket / esp_http_server)
  │
  ├─ Assembles fragmented WebSocket frames into complete messages
  │  (connection.cpp: prepare_receive_buffer_ / commit_receive_buffer_)
  │
  ├─ Checks message_dispatch_enabled_ atomic flag
  │  (returns immediately if disabled; used during teardown)
  │
  └─ Invokes callback on network thread:
     ├─ Text → SendspinClient::process_json_message_()
     └─ Binary → SendspinClient::process_binary_message_()

JSON Message Dispatch (network thread)

process_json_message() (src/client.cpp) parses the message type and routes:

Message Action on Network Thread
SERVER_HELLO Stores server info and connection reason on the connection, then sets server_hello_received_ (an atomic store that publishes the fields to the main loop; the manager's promotion scan observes is_handshake_complete() on its next tick)
SERVER_TIME Pushes a TIME_RESPONSE InboxEvent onto the shared inbox ring
SERVER_STATE Writes/merges into the controller, metadata, and color InboxSlots via each role's handle_server_state()
SERVER_COMMAND Merges into the player's command_slot (InboxSlot<ServerCommandMessage>)
GROUP_UPDATE Merges into Client::group_slot (InboxSlot<GroupUpdateObject>)
STREAM_START Writes to the player's stream_params_slot, pushes a PLAYER_STREAM (STREAM_START) event onto the inbox ring. Marks the artwork stream active, flushes the decode thread's notification queue, bumps the artwork stream_epoch, and resets the artwork display_slot. Writes the config to the visualizer's config_slot and pushes a VISUALIZER_STREAM (STREAM_START) event onto the inbox ring.
STREAM_END Pushes a PLAYER_STREAM (STREAM_END) event onto the inbox ring and signals sync task COMMAND_STREAM_END; pushes ARTWORK_STREAM (STREAM_END) and VISUALIZER_STREAM (STREAM_END) events onto the inbox ring
STREAM_CLEAR Pushes ARTWORK_STREAM (STREAM_CLEAR) and VISUALIZER_STREAM (STREAM_CLEAR) events onto the inbox ring; for the player, signals sync task COMMAND_STREAM_CLEAR and enqueues a CHUNK_TYPE_STREAM_CLEAR_MARKER chunk into the encoded ring buffer (no player listener callback - a seek is not a stream lifecycle event)

JSON parse arena (src/platform/json_arena.h)

The JsonDocument used to parse each incoming message comes from make_json_document(). By default that allocates the document's variant pool and copied strings out of PSRAM (PsramJsonAllocator), which puts PSRAM traffic on the CPU-hot network thread for every message. When SendspinClientConfig::json_arena_size > 0 (the default is 2048), SendspinClient instead owns a SendspinArenaAllocator (a fixed internal-RAM bump arena) and process_json_message() calls reset() on it and parses into a document backed by it. An allocation that does not fit the remaining budget falls back to platform_malloc (PSRAM-preferring), so an unexpectedly large message (e.g. track metadata) still parses, just slowly.

The bump arena suits ArduinoJson's allocation pattern: during a parse the variant pool is allocated once up front and the deserializer's string scratch buffer is always the most-recently-allocated block while it grows and shrinks, so those reallocations happen in place; document teardown frees strings newest-first and the pool last (LIFO), draining the arena back to empty on its own. reset() between messages is a safety net for any arena block left behind by a non-LIFO free; it cannot free PSRAM fallbacks (those are released by deallocate() on document teardown like any other allocation). The allocator is not thread-safe; the single instance is owned by SendspinClient and touched only on the network thread (process_json_message() runs serialized on the httpd worker task, and the previous call's JsonDocument is destroyed before the next call). Outgoing-message serialization in src/protocol.cpp still uses the PSRAM allocator.

Binary Message Dispatch (network thread)

process_binary_message() extracts the type byte and routes:

Binary Type Handler
Player audio PlayerRole::Impl::handle_binary(): writes to encoded audio ring buffer
Artwork image ArtworkRole::Impl::handle_binary(): copies image data to a per-slot double buffer and enqueues a notification for the artwork decode thread
Visualizer data (binary types 16-20) VisualizerRole::Impl::handle_binary(): writes to visualizer ring buffer

Main Loop Processing

SendspinClient::loop() (src/client.cpp) runs the following steps in order on each tick:

1. connection_manager_->loop()   (sections gated on lock-free atomic hints - see below)
   ├─ Start WS server if network ready
   ├─ Swap deferred connection events under mutex   (only when has_pending_events_)
   ├─ Process close/disconnect events (on_connection_lost)
   ├─ Promotion scan: establish nursery connections whose hello handshake completed
   │  (handoff decisions against the incumbent)
   ├─ Call loop() on the current and nursery connections   (only when has_current_ or nursery_size_)
   ├─ Check per-connection hello retry timers   (only when nursery_size_)
   ├─ Reap nursery connections past the establish deadline; tick the platform ws_server
   └─ flush_deferred_releases()   (early-returns without locking when deferred_size_ is 0)

2. time_burst_->loop(conn)  (skipped when no current connection)
   ├─ Send next time message if ready
   ├─ Acquire/release high-performance networking around burst
   └─ Notify listener of sync error when burst completes

3. Drain inbox event ring (when INBOX_TOPIC_EVENTS is set)
   ├─ Feed TIME_RESPONSE events into time_burst_->on_time_response()
   ├─ Fire CONTROLLER/METADATA/COLOR_CLEARED via each role's handle_cleared_event()
   ├─ Dispatch PLAYER_STREAM via player_->impl_->on_stream_ring_event()
   ├─ Dispatch ARTWORK_STREAM via artwork_->impl_->handle_stream_ring_event()
   └─ Dispatch VISUALIZER_STREAM via visualizer_->impl_->handle_stream_ring_event()

4. Role event draining (each role's impl_->drain_events(), gated on impl_->needs_drain(slot_bits))
   ├─ player_->impl_->drain_events()
   ├─ controller_->impl_->drain_events()
   ├─ metadata_->impl_->drain_events()
   ├─ color_->impl_->drain_events()
   └─ artwork_->impl_->drain_events()   (display-deadline sweep; visualizer has no drain_events())

5. Drain group_slot (when INBOX_TOPIC_GROUP is set in slot_bits)
   └─ Apply group deltas, fire on_group_update, persist last played server

This ordering matters: connection lifecycle events are processed before role events, and time sync before audio processing, so that roles always see a consistent connection and time state.

Each loop() section that would otherwise take a mutex first consults a lock-free atomic hint, so an idle tick pays only for the atomic loads it needs to decide there is nothing to do. ConnectionManager keeps four such hints, each refreshed under the owning mutex right after the container/pointer it mirrors changes (always re-derived from .size() or the assigned value, never incremented in place, so the hint cannot drift from ground truth):

  • has_pending_events_ (under conn_mutex_): set at every push into either deferred connection-event queue, cleared once loop() has swapped both out. Lets loop() skip the conn_mutex_ acquisition when neither queue has anything pending.
  • nursery_size_ (under conn_ptr_mutex_): mirrors nursery_.size(). Lets loop() skip the current/nursery copy-and-loop() block, the hello-retry scan, and the nursery reap scan when the nursery is empty, while keeping the lifecycle block running while any nursery connection exists (its promotion scan is level-triggered on connection flags, not on events).
  • has_current_ (under conn_ptr_mutex_): true whenever current_connection_ is non-null. Lets loop() skip the copy-and-loop() block when there is no current connection and the nursery is empty.
  • deferred_size_ (under conn_ptr_mutex_): mirrors deferred_releases_.size(). Lets flush_deferred_releases() early-return without locking when nothing is queued.

Steady state is therefore cheap: connected-and-idle costs one conn_ptr_mutex_ acquisition (the current/nursery copy ahead of the conn->loop() calls) plus a handful of atomic loads; disconnected-and-idle costs zero mutex acquisitions. The mutex-protected containers and pointers remain the ground truth in every case; the hints only decide whether it is worth locking to look.

The Inbox drain steps gate the same way, off two lock-free poll() snapshots of the topic bitmask. inbox_bits is taken first and gates only the event-ring drain (step 3). slot_bits is taken after that drain completes and gates the role drains (step 4) and the group drain (step 5); the second snapshot catches topic bits a producer set while the ring drain was running (including a ring event's own side effects re-entering the inbox). A bit either snapshot races and misses is not lost - it stays set and the next tick's poll() observes it (bounded staleness, per Inbox::poll()). Each role's needs_drain(slot_bits) decides whether its drain runs: mostly a simple slot_bits & INBOX_TOPIC_* test, but the player, metadata, and artwork roles OR in a main-thread-only carry-over term (the player's awaiting_sync_idle_events, the metadata role's future-dated held_delta, the artwork role's nonzero held_display_mask) so that work waiting out a deadline no inbox bit tracks still gets a drain every tick until it fires.

Role Event Draining

Most roles implement drain_events() to process their deferred state on the main loop thread; stream lifecycle events instead ride the shared inbox ring and are dispatched from the ring drain (step 3 above) via each role's handle_stream_ring_event() / on_stream_ring_event() / handle_cleared_event(). Together these convert cross-thread inbox writes into sequential, single-threaded callback delivery. (The visualizer role has no drain_events() - all its delivery is ring-driven.)

PlayerRole::Impl::drain_events() (src/player_role.cpp:363)

Three stages, processed in order:

1. Client state updates: Takes from state_slot (latest-wins InboxSlot, written by the sync task). Calls client_->update_state().

2. Server commands: Takes from command_slot. Checks each field independently (volume, mute, static_delay) and fires the corresponding listener callback.

3. Stream lifecycle: The most complex part:

PLAYER_STREAM ring events → on_stream_ring_event() → awaiting_sync_idle_events list
                       │
                       ▼
         For each event in order:
           ├─ STREAM_END:
           │    If sync task is still running → wait for next tick
           │    If sync task is idle → fire on_stream_end(), continue
           │
           └─ STREAM_START:
                Take stream_params_slot
                Mark stream active, fire on_stream_start()
                Signal sync task COMMAND_START

The awaiting_sync_idle_events list (on PlayerRole::Impl) is the key ordering mechanism. STREAM_END callbacks are held until the sync task has reached its IDLE state, preventing the main loop from processing a new STREAM_START before the sync task has finished with the old stream. Events ahead of the blocked event also wait, preserving FIFO order. (stream/clear is not queued here — it is handled synchronously in handle_stream_clear() by signaling the sync task and enqueuing a marker chunk.)

Other Roles

  • ControllerRole: Takes the latest ServerStateControllerObject from its InboxSlot, fires on_controller_state(). The disconnect clear is not handled here; it arrives as a CONTROLLER_CLEARED event on the shared ring, whose handle_cleared_event() fires on_controller_state_clear() (deferred from cleanup() to avoid invoking the listener while ConnectionManager holds conn_ptr_mutex_).
  • MetadataRole: InboxSlot has no take_if, so the deadline gate that used to run under the shadow slot's mutex is split in two: take() unconditionally moves any pending delta into a main-thread-only held_delta (folding it into an already-held delta), then the server-clock deadline is evaluated with no lock held, applying deltas and firing on_metadata() once the timestamp is reached (or immediately if there is no active connection). A future-dated held_delta persists across ticks with no topic bit set, which is why needs_drain() ORs in held_delta.has_value() alongside the INBOX_TOPIC_METADATA bit test: the deadline sets no inbox bit, so without that term a bit-gated tick would strand the delta until an unrelated new delta happened to re-set the bit, silently starving deadline-based delivery. The clear arrives separately as a METADATA_CLEARED ring event (handle_cleared_event() fires on_metadata_clear()), deferred from cleanup() for the same conn_ptr_mutex_ reason.
  • ColorRole: Same structure as MetadataRole: take() into held_delta, a lock-free server-clock deadline gate firing on_color(), and a COLOR_CLEARED ring event driving on_color_clear().
  • ArtworkRole: Stream end/clear lifecycle is handled earlier in the tick by handle_stream_ring_event() (dispatched from the ring drain, before this call), which clears held_display_mask/display_slot and fires on_image_clear() for each configured slot - preserving the "lifecycle before display" ordering the old single-function drain guaranteed. drain_events() itself folds any taken display_slot update into the main-thread-only held_display_* state (latest-wins per slot), then sweeps the held slots and fires on_image_display(slot, lateness_ms) for any whose timestamp is due on the synced client clock (or immediately if there is no active connection). The deadline is computed by the pure display_overdue_us() helper, which applies the slot's display_offset_ms shift (positive fires early) and returns the overdue microseconds; display_lateness_ms() maps that to the lateness_ms argument, reserving 0 for the no-connection case (a connected on-time display is floored to 1 ms so it never collides with that sentinel). Per-slot epochs drop a held display whose stream was replaced after the decode hand-off. needs_drain() ORs a nonzero held_display_mask into the INBOX_TOPIC_ARTWORK_DISPLAY bit test (the same carry-over pattern the metadata role uses for held_delta) so held displays keep getting a drain every tick until their deadline fires, even though the deadline sets no inbox bit; on_image_decode still happens on the dedicated artwork decode thread.
    • Ack gate (require_frame_done): A slot can opt into per-slot back-pressure. Each SlotBuffer carries a SlotAckState (IDLE -> DECODE_DELIVERED once on_image_decode() fires -> PRESENTED once on_image_display()/on_image_clear() fires), all guarded by slot_mutex. While a gated slot is not IDLE, the decode thread (process_notification()) does not decode a newer notification; it parks it latest-wins in SlotBuffer::parked (has_parked) instead of decoding concurrently with the un-acked delivery. ArtworkRole::frame_done(slot) (main loop) returns the gate to IDLE and, if a notification is parked, calls wake_drain_thread() -- a sentinel ARTWORK_RECHECK_SLOT notification that unblocks the decode thread's notify_queue.receive() so it re-runs the top-of-loop parked-slot sweep (a dropped wake is covered by the DRAIN_RECEIVE_TIMEOUT_MS fallback). The parked notification is re-validated on replay, so a since-stale generation/epoch is simply skipped. A clear counts as a delivery: handle_stream_ring_event() drops any parked notification and forces gated slots to PRESENTED, so exactly one frame_done() is owed after it. A stream restart releases only DECODE_DELIVERED slots (their display can no longer fire); PRESENTED stays armed because the consumer may still be mid-fade on the prior stream's last delivery. There is no timeout.
  • VisualizerRole: Has no drain_events(). STREAM_START/END/CLEAR are dispatched entirely from handle_stream_ring_event() (from the ring drain): STREAM_START take()s the config from config_slot and fires on_visualizer_stream_start(); STREAM_END/CLEAR fire on_visualizer_stream_end()/on_visualizer_stream_clear().

Sync Task State Machine

The sync task (SyncTask::thread_entry, src/sync_task.cpp) runs a two-level state machine on its dedicated thread.

Outer Loop (per-stream lifecycle)

┌──────────────────────────────────────────────────────────┐
│                    COMMAND_STOP?                          │
│                    ┌─── yes ──→ exit thread               │
│                    │                                      │
│  ┌─────────────────┴──────────────────┐                  │
│  │           IDLE STATE               │                  │
│  │  • Clear TASK_RUNNING and all      │                  │
│  │    COMMAND flags                   │                  │
│  │  • Set TASK_IDLE                   │                  │
│  │  • Reset context + progress queue  │                  │
│  │  • Wait for codec header (500ms)   │◄──┐              │
│  └────────────┬───────────────────────┘   │              │
│               │ got header                │              │
│               ▼                           │              │
│  ┌────────────────────────────────────┐   │              │
│  │     WAIT FOR CLIENT ACK            │   │              │
│  │  • Wait on COMMAND_START or        │   │              │
│  │    STOP/END/CLEAR                  │   │              │
│  │  • If END/CLEAR arrives, return    │───┘              │
│  │    header to buffer and loop back  │                  │
│  └────────────┬───────────────────────┘                  │
│               │ COMMAND_START                             │
│               ▼                                          │
│  ┌────────────────────────────────────┐                  │
│  │         ACTIVE STATE               │                  │
│  │  • Clear TASK_IDLE, COMMAND_START  │                  │
│  │  • Drain stale playback progress   │                  │
│  │  • Set TASK_RUNNING                │                  │
│  │  • Enqueue SYNCHRONIZED state      │                  │
│  │  • Decode initial codec header     │                  │
│  │  • Run inner state machine loop    │                  │
│  └────────────┬───────────────────────┘                  │
│               │ STOP/END/CLEAR                           │
│               ▼                                          │
│  ┌────────────────────────────────────┐                  │
│  │  Return borrowed ring buffer entry │──────→ loop back │
│  └────────────────────────────────────┘                  │
└──────────────────────────────────────────────────────────┘

The WAIT FOR CLIENT ACK step is critical. Without it, the sync task could race from IDLE back to ACTIVE so fast that the main loop never observes TASK_IDLE, and the awaiting_sync_idle_events mechanism in PlayerRole::drain_events() would deadlock waiting for an idle transition that already passed.

Inner State Machine (active stream)

INITIAL_SYNC ──→ LOAD_CHUNK ──→ SYNCHRONIZE_AUDIO ──→ TRANSFER_AUDIO
     │                ▲               │                       │
     │                └───────────────┴───────────────────────┘
     │                        (cycle per chunk)
     └──→ LOAD_CHUNK (once first playback progress callback confirms frames were consumed)

INITIAL_SYNC: Fills the audio pipeline with silence to prime DMA buffers. Sleeps briefly after sending to let the audio stack start consuming. Once the first playback-progress callback confirms frames were consumed, it queues extra_startup_silence_ms of additional silence (see PlayerRoleConfig) and drains it before advancing to LOAD_CHUNK. This extra lead gives the decode pipeline slack to stay ahead of the sink at stream start, preventing the initial-playback stutter caused by the decoder briefly falling behind.

LOAD_CHUNK: Reads the next encoded chunk from the ring buffer. Waits for time sync if not yet available. Decodes audio via FLAC/Opus/PCM decoder. On a ring-buffer underflow (no chunk ready) while still aligning (startup or post-seek), it feeds silence toward the sink to keep the DAC fed while the decode pipeline catches up, instead of letting it run dry; SYNCHRONIZE_AUDIO then re-aligns the next chunk against wherever the silence carried us. In steady state it does not fill — an empty buffer there means the stream is winding down, and stuffing silence would pile up in the sink and delay a rapid restart (a genuine underrun instead surfaces as an error in SYNCHRONIZE_AUDIO).

SYNCHRONIZE_AUDIO: Computes the sync error:

error = decoded_timestamp - new_audio_client_playtime

Where decoded_timestamp is the server timestamp converted to client time (via Kalman filter) minus static and fixed delays, and new_audio_client_playtime is the predicted time that the next audio will actually play.

Error Range Action
> +5000 us (or +500 us settling) Hard sync ahead: insert silence frames to fill the gap
< -5000 us (or -500 us settling) Hard sync behind: drop the decoded chunk
+100 to +5000 us Soft sync: insert one interpolated frame near the end (average of last two)
-100 to -5000 us Soft sync: remove last frame (blend into second-to-last)
-100 to +100 us Dead zone: pass audio through unmodified

Hard sync sets a flag that switches to a tighter 500 us settle threshold until the error is small enough to exit hard sync mode.

TRANSFER_AUDIO: Writes PCM data to the audio sink via on_audio_write. If silence was inserted (hard sync ahead), transfers silence first, then re-enters SYNCHRONIZE_AUDIO for the held-back decoded data.

Playback Progress Tracking

The audio output hardware reports consumed frames via notify_audio_played() → playback_progress_slot_ (a ShadowSlot whose merge strategy sums frames_played across unread updates and keeps the latest finish_timestamp). The sync task takes the accumulated value on every inner loop iteration to maintain an accurate new_audio_client_playtime estimate:

new_audio_client_playtime = last_finish_timestamp + remaining_buffered_frames_as_microseconds

This feedback loop is what makes the sync error calculation accurate.

Time Synchronization

Burst Strategy (src/time_burst.h)

Time sync uses a burst-based NTP-style protocol:

  1. Send 8 time request messages per burst (each with a 10-second response timeout).
  2. Wait 10 seconds between bursts.
  3. Select the measurement with the lowest round-trip time (lowest max_error).
  4. Feed the best measurement into the Kalman filter.

High-performance networking (e.g., disabling WiFi power saving) is acquired for the duration of a burst and released when complete.

Kalman Filter (src/time_filter.h)

Two-dimensional state vector: [offset, drift].

  • First measurement establishes the offset baseline.
  • Second measurement estimates initial drift from finite differences.
  • Subsequent measurements: predict offset forward by drift * dt, then correct using the new measurement.
  • Adaptive forgetting: if the residual exceeds 3.0 * max_error, the covariance is inflated by a forgetting factor (2.0) to recover from step changes.
  • Drift compensation is only enabled after 100 samples and only when drift significance exceeds its noise floor.

The filter is protected by state_mutex_ so that compute_client_time() can be called from the sync task thread while update() runs from the main loop thread.

Connection Lifecycle

Connection Management (src/connection_manager.cpp)

The ConnectionManager maintains one established slot plus a bounded nursery of unproven connections:

Slot Purpose
current_connection_ Active connection receiving messages; holds only connections that completed the hello handshake
nursery_ Unproven connections (inbound or outbound) awaiting establishment, bounded by NURSERY_CAPACITY inbound + 1 outbound

All are std::shared_ptr<SendspinConnection>, and on the ESP server path they are observers rather than authoritative owners; the authoritative owner of a SendspinServerConnection is the httpd session itself (see Server connection ownership (ESP)). On the host (IXWebSocket) client path the shared_ptr in these slots is the only owner.

Handshake and Handoff

  1. A new connection (outbound or inbound) enters the nursery. Inbound connections are delivered by the platform ws_server only after their WebSocket upgrade is observed.
  2. The connection sends a CLIENT_HELLO. Retry with exponential backoff (100 ms base, 3 attempts). Each managed connection has its own retry entry in ConnectionManager::hello_retries_, so a handoff candidate arriving mid-handshake cannot clobber another connection's pending hello.
  3. The handshake state lives on the connection as two atomic flags: client_hello_sent_ (set by the hello send-completion callback) and server_hello_received_ (set when SERVER_HELLO is processed on the network thread, after the server info fields it publishes).
  4. Establishment is level-triggered: each loop() tick, the promotion scan promotes any nursery connection whose is_handshake_complete() is true, so the order in which the two flags were set is irrelevant. If an incumbent exists, the handoff decision runs:
    • Prefer PLAYBACK reason over DISCOVERY.
    • Among two DISCOVERY connections, prefer the last-played server.
    • Default: keep current.
  5. Handoff executes: disable the loser's message dispatch → cleanup client state (winner only gets on_handshake_complete) → send goodbye to the rejected connection via the deferred-release queue.

Disconnection and Cleanup

When a connection is lost (on_connection_lost):

1. conn->disable_message_dispatch()      ← atomic, immediate on network thread
2. client_->cleanup_connection_state()  ← stop time sync, reset the inbox event ring and every
                                           role's slots, signal stream end
3. Queue the connection on deferred_releases_ (released outside the manager lock)
4. The current slot stays empty; the next promotion scan fills it from the nursery

disable_message_dispatch() is the first step because it's an atomic flag that the network thread checks before invoking any callback. This prevents stale messages from a dead connection from racing into freshly-reset role queues.

Graceful Disconnect

disconnect_and_release() calls conn->disconnect(reason, nullptr) and lets the local shared_ptr go out of scope.

  • ESP server: the goodbye text is queued as an httpd worker job. The worker resolves the connection by lock()ing the weak_ptr captured in the queued arg when the goodbye was enqueued; if it resolves it sends the frame, then runs the completion lambda that calls trigger_close(). The session slot installed in open_callback keeps the connection alive across that whole sequence even after ConnectionManager's observer shared_ptr is dropped. The session is finally freed when httpd invokes the slot's free_fn (see Server connection ownership (ESP)). The completion lambda also captures a weak_ptr to make this lifetime explicit — trigger_close() is skipped if the conn has already been freed. Goodbye is one of the two messages that pass allow_before_hello=true, so it is not blocked by the pre-hello send gate (a rejected connection is told to leave before it ever sends a hello).
  • Host client: the IXWebSocket send is synchronous, so the goodbye and close have both completed by the time disconnect() returns and the shared_ptr drops the last reference.

Server connection ownership (ESP)

On the ESP build, SendspinServerConnection lifetime is pinned to the httpd session rather than to ConnectionManager:

  1. SendspinWsServer::open_callback (the httpd open_fn) creates the shared_ptr<SendspinServerConnection>, heap-allocates a shared_ptr* slot, and calls httpd_sess_set_ctx(handle, sockfd, slot, free_fn) with a deleter that deletes the slot. That slot is the authoritative reference.
  2. The same shared_ptr is forwarded into ConnectionManager::on_new_connection, which admits it into the nursery as a secondary observer (it moves to current_connection_ only once its hello handshake completes).
  3. The httpd WebSocket handler (websocket_handler) looks the connection up by httpd_sess_get_ctx(handle, sockfd) at run time, copying the slot's shared_ptr for the duration of its work; it never assumes the manager's observer slot is alive. The queued send workers (async_send_text, async_send_time_text) instead capture a weak_ptr<SendspinServerConnection> to the originating connection and lock() it when they run.
  4. When the socket closes, httpd calls the close_fn first (which fires connection_closed_callback_ so ConnectionManager can drop its observer in the next loop()), then later calls the slot's free_fn to release the authoritative reference once no workers are queued for that session.

Queued send workers capture a weak_ptr<SendspinServerConnection> to the originating connection — AsyncRespArg for text sends, SessionLookup for time sends — and lock() it when the worker runs. This is deliberately not a {httpd_handle_t, int sockfd} pair: identifying the target by sockfd risked binding to a different connection that had recycled the same fd after the original closed, sending a frame to the wrong peer. The weak_ptr resolves to the exact connection that queued the work, or to null if it has since been destroyed, in which case the worker no-ops cleanly. Because these structs now hold non-trivial members (the weak_ptr, and AsyncRespArg's completion std::function), they are constructed with placement-new and explicitly destroyed before platform_free rather than treated as POD. Both are allocated through platform_malloc / platform_malloc_internal.

The send workers also enforce the protocol's "hello is always first" rule: a frame is dropped unless client_hello_sent_ is set on the resolved connection, unless the caller passed allow_before_hello=true. Exactly two callers do — the client/hello itself (which would otherwise gate its own send and deadlock) and goodbye — so a stale or out-of-order frame can never precede the handshake. The weak_ptr guards identity; the gate guards ordering; the two are independent.

The host build does not need this scheme: SendspinWsServer (host) routes IXWebSocket messages by calling find_connection_callback_ to resolve a synthetic sockfd back to the connection that ConnectionManager is holding. The ESP build keeps the set_find_connection_callback() setter as a no-op stub for symmetry; see the comment at the call site in ConnectionManager::init_server.

Ordering Guarantees Summary

Network Thread → Main Loop

All network thread actions are deferred to the main loop, primarily through the shared Inbox (its event ring and InboxSlots), with a few remaining per-thread queues and mutex-protected vectors. The main loop processes them in a fixed order each tick (connections → time → roles → group). This guarantees that:

  • Connection state is settled before roles process events.
  • Time sync is updated before audio sync decisions.
  • Role events fire in FIFO order per role.

Stream Lifecycle Ordering

The combination of awaiting_sync_idle_events (main loop) and COMMAND_START (sync task wait) creates a two-way handshake:

  1. Network thread pushes STREAM_END → STREAM_START as PLAYER_STREAM events onto the inbox event ring.
  2. Sync task receives COMMAND_STREAM_END, finishes active stream, enters IDLE, sets TASK_IDLE.
  3. Main loop drains the ring into awaiting_sync_idle_events, sees STREAM_END, checks is_running() → false (idle), fires on_stream_end().
  4. Main loop sees STREAM_START, fires on_stream_start(), signals COMMAND_START.
  5. Sync task receives COMMAND_START, exits wait, enters ACTIVE.

This prevents the sync task from starting a new stream before the main loop has processed the end of the old one.

Playback Progress

Audio output callbacks run on a platform audio thread. They report consumed frames via notify_audio_played() → playback_progress_slot_ (merging sums frames and keeps the latest timestamp). The sync task takes the accumulated value non-blockingly on each iteration of its inner loop, keeping the playtime estimate accurate without blocking the audio thread.

Cleanup Atomicity

disable_message_dispatch() + queue draining + event flag signaling ensures that after cleanup:

  • No new messages will be delivered from the old connection.
  • All pending events are discarded.
  • The sync task is signaled to end its current stream.
  • The main loop will process the synthetic STREAM_END on its next tick.

Re-entrant Teardown During Callback Dispatch

A listener callback fired from the main loop can synchronously re-enter connection teardown - for example an on_stream_start() or on_*_clear() handler that calls connect_to(), whose drop_connection() runs cleanup_connection_state() on the same stack. That cleanup wipes the inbox event ring (reset_events()) and each role re-pushes its synthetic clear/STREAM_END, so a drain already in progress must not act on the stale events it copied out before the wipe. Two plain uint32_t generation counters, both touched only on the main loop, guard this:

  • SendspinClient::event_state_->drain_generation, bumped by cleanup_connection_state(). The event-ring drain in loop() snapshots it before the batch and aborts the instant it changes, discarding the events it had already copied out (they were wiped deliberately) and leaving the cleanup's freshly re-pushed events in the live ring for the next tick. The abort is checked both at the top of the inner per-event loop and once more after the loop body, so a teardown that re-enters on the final event of a full batch cannot fall through into a second take_events() that would destructively pull, and then drop, those re-pushed events.
  • PlayerRole::Impl::cleanup_generation, bumped by the player's cleanup(). drain_events() snapshots it around each on_stream_start() call; if it changes, the stream was torn down from inside the callback, so the player abandons the rest of the batch rather than re-arm the sync task for a dead stream. stream_active is set before the callback runs, so the STREAM_END that cleanup() enqueued still passes its gate and delivers a paired on_stream_end().

Neither counter needs atomics: they only let an in-flight drain notice that teardown ran underneath it and stop touching state that cleanup already reset.