ASR/TTS Benchmark Suite

Compare ASR (Speech-to-Text) and TTS (Text-to-Speech) across 3 deployment options:

Option	Type	ASR	TTS	Setup	Best For
1. Local	Direct Python + CUDA	faster-whisper	Kokoro	Complex	Max performance, Jetson
2. Open WebUI	Docker + UI	Whisper	Built-in	Easy	Workshops, demos
3. Speaches	Docker API	faster-whisper	Kokoro	Easy	API integration

Supported Platforms

Platform	Option 1 (Local)	Option 2 (Docker)	Option 3 (Docker)
Windows + NVIDIA GPU	Yes (CUDA)	Yes	Yes
Linux + NVIDIA GPU	Yes (CUDA)	Yes	Yes
Jetson Thor	Yes (CUDA)	Limited	Limited
Jetson Orin	Yes (CUDA)	Limited	Limited

Directory Structure

comp/
├── README.md                  # This file
├── run_all_benchmarks.py      # Compare all 3 options
│
├── shared/                    # Shared resources
│   ├── benchmark_utils.py     # Common utilities
│   ├── test_audio/            # Test audio files
│   └── results/               # Benchmark results
│
├── option1_local/             # Direct Python execution
│   ├── README.md
│   ├── requirements.txt
│   └── benchmark.py
│
├── option2_openwebui/         # Docker Open WebUI
│   ├── README.md
│   ├── docker-compose.yml
│   └── benchmark.py
│
└── option3_speaches/          # Docker Speaches API
    ├── README.md
    ├── docker-compose.yml
    ├── docker-compose.gpu.yml
    └── benchmark.py

Quick Start

1. Setup Test Audio

# Generate synthetic test audio (or add your own .wav files)
python generate_test_audio.py

2. Run Comparison

python run_all_benchmarks.py

Individual Option Setup

Option 1: Local

cd option1_local
pip install -r requirements.txt
python benchmark.py

Option 2: Open WebUI

cd option2_openwebui
docker compose up -d
python benchmark.py

# Open UI: http://localhost:3000

Option 3: Speaches

cd option3_speaches
docker compose up -d
python benchmark.py

# API docs: http://localhost:8000/docs

Comparison Overview

                    LATENCY                         EASE OF USE
                    =======                         ===========
                    
   LOWEST ◄─────────────────────────► HIGHEST     COMPLEX ◄─────────► SIMPLE
   
   Option 1         Option 3         Option 2     Option 1   Option 3  Option 2
   (Local)          (Speaches)       (Open WebUI) (Local)    (API)     (UI)
   
   ~200-400ms       ~250-500ms       ~300-600ms   pip+CUDA   docker    docker
   for 5s audio     (+25% overhead)  (+50% overhead)

When to Use Each Option

Option 1: Local

• Best for: Maximum performance, offline use, embedded apps
• Latency: Lowest (no network overhead)
• Setup: Complex (Python deps, CUDA)
• API: Direct Python function calls

Option 2: Open WebUI

• Best for: Workshops, demos, non-technical users
• Latency: Highest (web interface overhead)
• Setup: Easy (single docker command)
• API: Web-based, not programmatic

Option 3: Speaches

• Best for: API integration, microservices, OpenAI SDK compatibility
• Latency: Medium (~15-25% overhead vs local)
• Setup: Easy (single docker command)
• API: OpenAI-compatible REST API

Expected Results

x86_64 (Windows/Linux with NVIDIA GPU)

Test	Local (CUDA)	Speaches	Open WebUI
ASR 5s	200-400ms	250-500ms	300-600ms
ASR 30s	1000-2000ms	1200-2400ms	1500-3000ms
TTS short	100-300ms	150-400ms	200-500ms

Jetson Thor (Blackwell GPU)

Test	Model	CPU	CUDA
ASR 5s	base	800ms	150ms
ASR 5s	large-v3	4000ms	400ms
ASR 30s	base	4000ms	600ms
TTS short	kokoro	500ms	100ms

Thor's Blackwell GPU provides 5-10x speedup over CPU.

Requirements

• Python 3.11+ (3.12 recommended for full local TTS)
• Docker Desktop (for Options 2 & 3)
• NVIDIA GPU (optional, for faster processing)

References

• Speaches - OpenAI-compatible ASR/TTS
• Open WebUI - Self-hosted AI chat
• faster-whisper - Fast Whisper
• Kokoro - #1 ranked TTS