Voice&Music Assistant

ThirdReality Voice&Music Assistant is an open-source speaker that supports connecting to the Home Assistant Voice Assistant and Music Assistant. You need to have a device running Home Assistant in order to use this speaker. If you do not have Home Assistant installed yet, refer to the installation documentation for instructions. Buy it on ThirdReality Shop

Architecture

The firmware consists of two main application components:

• Voice — built on linux-voice-assistant-cpp, a C++ rewrite of OHF-Voice/linux-voice-assistant. Implements the ESPHome native API so Home Assistant discovers the speaker as a voice satellite. See its README for details.
• Music — built on Sendspin, a lightweight streaming protocol integrated with Music Assistant.

---

• Voice&Music Assistant
  • Architecture
  • Build
    • Docker Build
    • Native Build
  • Flash
  • Debugging
    • Serial
    • ADB
    • SSH
  • Setup the voice assist
  • Setup through HA APP
  • Smart Home control with voice
  • Smart Home control with button
  • Play Music
  • Multi-Room Music
    • Work with Apple HomePod
    • Work with Sonos

---

Build

Clone the repository:

git clone https://github.com/thirdreality/voice-music-assistant.git
cd <YOUR PATH>/voice-music-assistant
git submodule update --init

Docker Build

No host dependencies required other than Docker.

./go --docker trspk <version>          # Build inside Docker (recommended)
./go --docker-shell                    # Enter container interactively for debugging
./go --docker trspk rebuild <package>  # Rebuild a single package in Docker

Native Build

Requires:

• Ubuntu 20.04

Install dependencies:

sudo apt-get update

sudo apt-get install -y build-essential bash bc binutils build-essential bzip2 cpio g++ gcc git gzip locales libncurses5-dev libdevmapper-dev libsystemd-dev make mercurial whois patch perl python rsync sed tar vim unzip wget bison flex libssl-dev libc6:i386 libncurses5:i386 libstdc++6:i386 zlib1g-dev:i386 zip python3-pip pkg-config automake gsettings-ubuntu-schemas libglib2.0-dev gcc-multilib g++-multilib

pip install pycrypto

wget http://ftp.cn.debian.org/debian/pool/main/a/automake-1.16/automake_1.16.1-4_all.deb && sudo dpkg -i automake_1.16.1-4_all.deb && rm -f automake_1.16.1-4_all.deb

Build:

./go trspk <version>               # If no version number is specified, the date will be used
./go trspk rebuild <package>       # Rebuild a single package

The generated image is located at:

<YOUR PATH>/voice-music-assistant/image

Flash

1. Download and extract Aml_Burn_Tool.zip

2. If this is your first time using the tool, click on Setup_Aml_Burn_Tool_V3.1.0.exe to install necessary drivers.

3. Next, navigate to the v2 folder and run Aml_Burn_Tool.exe.

4. Load the compiled **.img firmware file. Or you can download the latest firmware here.

5. Click on Start to initiate the burn process.

6. Use debug board to connect the speaker to the PC. If you don’t have a debug board, you can use a Type-C data cable. Make sure to use a data cable. Then power it on.

Debugging

Serial

1. Use debug board to connect the speaker to the PC. Make sure to use data cables.

2. Open your serial debugging tool, select the corresponding port, and set the baud rate to 115200.

ADB

ADB is enabled on development images and starts automatically at boot (/etc/init.d/S55adbd). It listens over USB and, by default, also over TCP on port 5555.

Warning: the TCP socket binds to 0.0.0.0 with no authentication. Anyone on the same network can obtain a root shell. To disable TCP, set ADB_TCP_PORT= in /etc/default/adbd (USB stays available).

USB — connect a Type-C data cable to the PC:

adb devices
adb shell

TCP — over the network (device and PC on the same LAN):

adb connect <device-ip>:5555
adb shell

Multiple devices — when more than one device is connected, adb shell fails with more than one device/emulator. You must first list the devices and then target one explicitly with -s <serial>. The serial differs by transport:

• USB devices are listed by their MAC address (for example a1b2c3d4e5f6).
• TCP devices are listed as <ip>:5555 (for example 10.1.0.33:5555).

$ adb devices
List of devices attached
a1b2c3d4e5f6    device        # USB, serial is the MAC address
10.1.0.33:5555  device        # TCP, serial is <ip>:5555

# pick the USB device by its MAC address
adb -s a1b2c3d4e5f6 shell

# pick the TCP device by its <ip>:5555
adb -s 10.1.0.33:5555 shell

Note: the same speaker connected over both USB and TCP shows up as two separate entries with different serials.

SSH

ssh root@<device-ip>

• Username: root
• Password: hello3r

Setup the voice assist

There are two ways to use the voice assistant: Home Assistant Cloud or local voice recognition (If your device doesn't have sufficient performance, please choose Home Assistant Cloud)

After completing either of the above options, add an assistant under Settings → Voice Assistants.

• Home Assistant Cloud

  Open the Home Assistant app or the Home Assistant web interface, Go to Settings → Home Assistant Cloud. Create or log in to your account. (30 day free trial)

  

• Local voice recognition

  Please refer to:

  https://github.com/rhasspy/wyoming-piper

  https://github.com/rhasspy/wyoming-faster-whisper

---

Setup through HA APP

1. You need to install the iOS or Android version of the Home Assistant app first. And please make sure the app is up to date.
2. Make sure the speaker is in a yellow blinking state. Otherwise, please try factory reset. (Press and hold the Home button for 15 seconds, then release it after you hear the prompt sound)
3. Open the Home Assistant app on your phone. Go to Settings → Devices & services and under Discovered, you should see the device as "3RSPK-XXXXX Improv via BLE". (If the device is not found, please check whether Bluetooth and Nearby Devices permissions are enabled in the app)

4. Enter your Wi-Fi SSID and password. Only 2.4 GHz networks are supported.

5. A few seconds after the Wi-Fi connection is successful, the speaker will play "Your device is ready to connect to Home Assistant." Go to Settings → Devices & services and under Discovered, you should see the device as "3RSPK-XXXXXXXXXXXX ESPHome".

6. Add device

7. Select the voice assistant you created in step 1 (Setup the voice assist)

Now you can try waking the device with "OK Nabu" and start a conversation. You can check the device status in Settings → Devices & Services → ESPHome.

---

Smart Home control with voice

Supported voice commands: https://www.home-assistant.io/voice_control/builtin_sentences/

• For example, "What's the time" or "Turn on the light in the living room".
• Make sure you're using the area name exactly as you defined it in Home Assistant.

Is the device you want to control via Assist (for example a specific light) not responding to your voice commands? Make sure the device is exposed to Assist: https://www.home-assistant.io/voice_control/voice_remote_expose_devices/

---

Smart Home control with button

We can create automation scripts based on the speaker's Home button trigger events to control devices. Supports single-click, double-click, and triple-click actions.

Settings → Devices & Services → ESPHome → Your device → Automations

---

Play Music

We can use Music Assistant to play music.

Settings → Add-ons → search for and add Music Assistant. After adding it, you can access it via port 8095.

If it doesn’t exist, please add it using Music Assistant github repository

Go to the Music Assistant: Settings → Player Providers → Add a player provider. Search for Sendspin and add it.

Then you can go to Settings → Players and find the player named 3RSPK-XXXXXXXXXXXX.

If it doesn’t show up, try reloading Sendspin.

Next, we need to add music sources. Go to Settings → Music Sources → Add a music source, then select your desired music source.

Finally, select a song and the speaker, and you can start playback! 🎵

---

Multi-Room Music

Official documentation: https://www.music-assistant.io/faq/groups/#groups

We use Sendspin as the playback protocol, so it can work with any device that supports AirPlay.

Work with Apple HomePod

Go to the Music Assistant: Settings → Player Providers → Add a player provider. Search for and add AirPlay and Sync Group Player.

Then you can create a sync group player with your thirdreality speaker and HomePod.

Work with Sonos

Go to the Music Assistant: Settings → Player Providers → Add a player provider. Search for and add SONOS and Universal Group Player.

Then you can create a universal group player with your thirdreality speaker and Sonos.