Skip to content

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

4 Commits

Folders and files

Repository files navigation

SiMa Liquid Audio Demo

This repo contains everything needed to run the SiMa liquid-audio model on SiMa Modalix.

It includes:

  • the local web app
  • install and run scripts
  • example request scripts for direct backend usage

The app supports:

  • ASR
  • TTS
  • Interleaved (Speech2Speech)

Runtime architecture:

  • browser <-> app.py via Flask + Socket.IO
  • app.py <-> sima_lmm backend via HTTP/SSE

app.py consumes those SSE streams server-side and forwards the streamed text/audio updates to the browser over Socket.IO.

Install

To run this demo, you need these three things in one workspace folder:

  • this repo: sima-liquid-audio/
  • the sima_lmm wheel
  • the compiled model folder

Recommended setup layout:

liquid_audio/
  sima-liquid-audio/
  sima_lmm-<version>.whl
  <compiled-model-folder>/

1. Create a workspace folder

mkdir -p liquid_audio
cd liquid_audio

2. Clone this repo into it

git clone git@github.com:florianvoss-commit/sima-liquid-audio.git

3. Download the sima_lmm wheel into liquid_audio/

Download the wheel from:

and place it here:

liquid_audio/sima_lmm-2.1.0-cp311-cp311-linux_aarch64.whl

4. Clone the compiled model into liquid_audio/

The compiled model should sit next to sima-liquid-audio, not inside it.

Example:

cd ..
git clone https://huggingface.co/florianvoss/LFM2.5-Audio-1.5B_compiled

Example result:

liquid_audio/
  sima-liquid-audio/
  sima_lmm-2.1.0-cp311-cp311-linux_aarch64.whl
  LFM2.5-Audio-1.5B_compiled/

5. Run the app installer

cd sima-liquid-audio
./install.sh

What this does:

  • creates a local virtual environment in .venv
  • installs Python dependencies from requirements.txt
  • installs the latest parent-directory sima_lmm-*.whl

Important:

  • install.sh expects a built sima_lmm wheel to exist in the parent directory
  • the venv is created with --system-site-packages
  • the script force-reinstalls the wheel into the app venv so it is used even if another sima_lmm version exists globally

Run

Show help:

./run.sh --help

Current usage:

Usage: ./run.sh [options] [-- app.py options]

Modes:
  (default)           Start backend + frontend
  --frontend-only     Start only frontend app.py
  --backend-only      Start only sima_lmm backend

Options:
  -h, --help              Show this help

Behavior:

  • run.sh auto-detects a compiled model in the parent directory
  • backend startup uses:
python -m sima_lmm.devkit.devkit_demo run <MODEL_PATH> --mode web
  • combined mode prompts for sudo first, then starts backend and frontend
  • app.py always talks to the local backend at 127.0.0.1:9998
  • --frontend-only does not require a local model checkout and is useful if the backend is already running on the same device

Typical Flows

Start backend + frontend:

./run.sh

Start only frontend against an existing backend:

./run.sh --frontend-only

Start only backend:

./run.sh --backend-only

UI Modes

The current UI supports:

  • ASR
  • TTS
  • Interleaved (Speech2Speech)

Notes:

  • ASR streams transcript updates into the chat UI
  • TTS streams audio chunks from the backend and plays them in the browser
  • interleaved sends microphone audio to /v1/realtime and streams both text and audio back
  • the editable prompt in the UI is currently sent to interleaved mode as ordinary text input, not as a privileged backend system prompt

Conversation State

For interleaved mode:

  • when Include chat history is enabled, app.py sends include_chat_history=true
  • the backend preserves interleaved conversation state across turns
  • /clear-history resets both frontend history and backend interleaved state
  • /stop stops the active request; for interleaved runs this also resets backend continuation state

Example Request Scripts

This repo includes small example scripts for direct backend interaction:

Supported TTS voices:

  • uk_female
  • uk_male
  • us_female
  • us_male

Examples:

python examples/asr_request.py --audio-path /path/to/audio.wav
python examples/tts_request.py --input "Hello there" --voice uk_female
python examples/interleaved_request.py --audio-path /path/to/audio.wav
python examples/interleaved_request.py --audio-path /path/to/audio.wav --text "Keep it short." --include-chat-history
python examples/interleaved_request.py --audio-path /path/to/audio.wav --no-include-chat-history
python examples/reset_history.py
python examples/stop.py

Optional:

python examples/asr_request.py --board-ip 192.168.1.10 --audio-path /path/to/audio.wav
python examples/tts_request.py --board-ip 192.168.1.10 --input "Hello there" --voice us_male
python examples/interleaved_request.py --board-ip 192.168.1.10 --audio-path /path/to/audio.wav
python examples/reset_history.py --board-ip 192.168.1.10
python examples/stop.py --board-ip 192.168.1.10

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages