feat(voxtral): Voxtral Realtime STT provider with concurrent streaming

.env.example

@@ -11,7 +11,7 @@ REDIS_HOST=127.0.0.1
 REDIS_PORT=6789
 REDIS_PASSWORD=
 
-# STT Provider: "gladia" (default) or "openai"
+# STT Provider: "gladia" (default), "openai" or "voxtral-realtime"
 # STT_PROVIDER=gladia
 
 # =============================================================================
@@ -81,3 +81,44 @@ GLADIA_TRANSLATION_LANG_MAP="de:de-DE,en:en-US,es:es-ES,fr:fr-FR,hi:hi-IN,it:it-
 
 # Base URL override — set this to use a compatible provider (e.g. a local Whisper server)
 #OPENAI_BASE_URL=
+
+# =============================================================================
+# --- Voxtral Realtime STT (STT_PROVIDER=voxtral-realtime) ---
+# =============================================================================
+
+#VOXTRAL_API_KEY=
+#VOXTRAL_MODEL=mistralai/Voxtral-Mini-4B-Realtime-2602
+#VOXTRAL_BASE_URL=
+
+# Silero VAD: minimum silence duration (s) before end-of-speech fires (default: 0.6)
+#VOXTRAL_VAD_MIN_SILENCE_S=0.6
+
+# Silero VAD: speech probability activation threshold 0–1 (default: 0.5)
+#VOXTRAL_VAD_ACTIVATION_THRESHOLD=0.5
+
+# Rolling audio pre-roll kept while idle and replayed when speech starts, so the
+# word onset preceding Silero's detection is not lost (default: 0.5)
+#VOXTRAL_VAD_PREROLL_S=0.5
+
+# Overlap replayed when a max-buffer split reopens mid-speech (default: 1.5).
+# The reopened request starts mid-utterance with no context; ~1.28 s of lead-in
+# is what the Voxtral paper recommends. The overlap is transcribed twice, so
+# some duplicated words at split boundaries are the trade-off for not losing them.
+#VOXTRAL_SPLIT_OVERLAP_S=1.5
+
+# Safety cap on a single streaming request before a forced split (default: 30.0).
+# Only reached during uninterrupted monologue. Splits replay VOXTRAL_SPLIT_OVERLAP_S
+# of audio, so lowering this mainly trades duplicated boundary words for faster
+# final transcripts during continuous speech (live interim captions are unaffected).
+#VOXTRAL_MAX_BUFFER_DURATION_S=30.0
+
+# How long to defer a segment's opening commit while the server is still
+# generating the previous segment's transcript (default: 10.0). vLLM silently
+# ignores commits sent during an in-flight generation, so the opener waits for
+# the previous transcription.done; audio is buffered locally and replayed, so
+# nothing is lost. On timeout (a done that never arrives) the pipeline resyncs
+# and opens anyway.
+#VOXTRAL_OPEN_GATE_TIMEOUT_S=10.0
+
+# Emit incremental transcription.delta events as interim captions (default: true)
+#VOXTRAL_INTERIM_RESULTS=true

.github/workflows/lint.yml

@@ -28,7 +28,7 @@ jobs:
 
       - uses: actions/setup-python@v6
         with:
-          python-version: "3.10"
+          python-version: "3.11"
 
       - name: Install the project
         run: uv sync --all-extras --dev

.github/workflows/publish-docker-image.yml

@@ -5,6 +5,7 @@ on:
   push:
     branches:
       - development
+      - voxtral-*
     tags:
       - "v*.*.*"
 

.github/workflows/tests.yml

@@ -24,7 +24,7 @@ jobs:
 
       - uses: actions/setup-python@v6
         with:
-          python-version: "3.10"
+          python-version: "3.11"
 
       - name: Install dependencies
         run: uv sync --group dev
@@ -54,7 +54,7 @@ jobs:
 
       - uses: actions/setup-python@v6
         with:
-          python-version: "3.10"
+          python-version: "3.11"
 
       - name: Install dependencies
         if: ${{ env.GLADIA_API_KEY != '' }}

.python-version

@@ -1 +1 @@
-3.10
+3.11

CHANGELOG.md

@@ -16,6 +16,7 @@ Final releases will consolidate all intermediate changes in chronological order.
 * build: declare aiohttp and numpy as direct dependencies
 * docs: correct how the OpenAI provider reaches the API and list its caveats
 * docs: expand and correct AGENTS.md
+* feat(voxtral): Voxtral Realtime STT provider with concurrent streaming and Silero neural VAD on Python 3.11
 
 ## v0.3.0
 

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.10-slim
+FROM python:3.11-slim
 
 # Use uv instead of pip, see https://github.com/astral-sh/uv
 RUN pip install --no-cache-dir uv==0.10.4

README.md

@@ -7,14 +7,15 @@ Supported STT engines:
 
 - **Gladia** — via the official [LiveKit Gladia plugin](https://docs.livekit.io/agents/integrations/stt/gladia/) (default)
 - **OpenAI** — via a direct REST client against `/v1/audio/transcriptions`; supports the official OpenAI API and any OpenAI-compatible endpoint
+- **Voxtral Realtime** — [Mistral Voxtral Mini Realtime](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602) served by a self-hosted [vLLM](https://docs.vllm.ai/) instance via its realtime WebSocket API
 
 ## Getting Started
 
 ### Environment prerequisites
 
-- Python 3.10+
+- Python 3.11+
 - A LiveKit instance
-- A Gladia API key **or** an OpenAI API key (depending on your chosen STT provider)
+- A Gladia API key, an OpenAI API key, **or** a vLLM server hosting Voxtral (depending on your chosen STT provider)
 - uv:
   - See installation instructions: https://docs.astral.sh/uv/getting-started/installation/
 
@@ -145,6 +146,29 @@ Some caveats apply to this provider:
   there is no detected language to map back to a BBB locale and the transcript is
   discarded with a warning. Set an explicit locale in BBB when using OpenAI STT.
 
+### Voxtral Realtime STT provider
+
+Set `STT_PROVIDER=voxtral-realtime` to use a self-hosted
+[Voxtral Mini Realtime](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602)
+model served by [vLLM](https://docs.vllm.ai/). The agent streams each
+participant's audio over vLLM's realtime WebSocket API, gated by a local
+Silero VAD, and emits live interim captions from the model's incremental
+deltas.
+
+```bash
+STT_PROVIDER=voxtral-realtime
+VOXTRAL_BASE_URL=https://your-vllm-server:8000/v1   # required
+VOXTRAL_API_KEY=your-key                            # if your server enforces one
+# VOXTRAL_MODEL=mistralai/Voxtral-Mini-4B-Realtime-2602  # default
+```
+
+VAD and segmentation tuning options (silence duration, pre-roll, split
+overlap, max segment length) are documented in `.env.example`.
+
+> **Note**: Voxtral Realtime does not support real-time translation. Only the
+> original transcript language is returned, matching the user's BBB speech
+> locale.
+
 ### Development
 
 #### Testing

providers/__init__.py

@@ -10,4 +10,11 @@ def create_agent(provider: str) -> BaseSttAgent:
         from providers.openai import OpenAiSttAgent, openai_config
 
         return OpenAiSttAgent(openai_config)
+    if provider == "voxtral-realtime":
+        from providers.voxtral_realtime import (
+            VoxtralRealtimeSttAgent,
+            voxtral_realtime_config,
+        )
+
+        return VoxtralRealtimeSttAgent(voxtral_realtime_config)
     raise ValueError(f"Unknown STT provider: {provider}")