Aviary

Run every good open-source TTS engine on your Mac, behind one switchboard.

Aviary is a self-hosted manager for local text-to-speech engines on Apple Silicon. One hub process (management plane + WebUI on :5050) supervises per-engine worker processes, each in its own isolated uv venv — because the engines' Python dependencies are mutually incompatible. Every worker speaks the same OpenAI-style HTTP contract on a stable port, so any client that can call one engine can call them all.

中文说明 → · License: MIT (code — model weights have their own licenses, see below)

• WebUI: start/stop/monitor engines (live logs, memory, RTF), generate and download speech, clone voices from reference audio, A/B-compare engines side by side, browse history. Desktop + mobile (PWA), light/dark.
• Engines are plugins: a folder with a manifest and a ~60-line adapter. The UI renders itself from each engine's self-reported capabilities — adding an engine changes zero UI code.
• Memory-aware: engines start on demand (a generation request auto-starts its engine) and unload after a configurable idle period.

Supported engines

All verified end-to-end (whisper transcription round-trip). RTF = synthesis time ÷ audio duration, measured on an Apple Silicon Mac; lower is faster, < 1 is faster than real time.

Engine	Official repo	Port	Runtime	Voice paradigm	RTF
VoxCPM2	OpenBMB/VoxCPM	5077	mlx-audio, 8-bit	voice description + cloning	~0.44
MOSS-TTS-Nano	OpenMOSS/MOSS-TTS-Nano	5075	mlx-audio	cloning (+ legacy /api/generate dialect)	~1.5
MeloTTS	myshell-ai/MeloTTS	5065	PyTorch CPU (Py3.11)	fixed presets (ZH mix-EN, EN accents)	~0.48
Qwen3-TTS (1.7B Base)	QwenLM/Qwen3-TTS	5051	mlx-audio, 8-bit	default voice + 3-second cloning¹	~0.24 warm
IndexTTS2	index-tts/index-tts	5052	mlx-indextts, 8-bit	cloning + 8-emotion control	~1.4
GPT-SoVITS (v2ProPlus)	RVC-Boss/GPT-SoVITS	5053	official api_v2.py, PyTorch CPU (Py3.10)	cloning (fine-tune offline)	~0.39
F5-TTS	SWivid/F5-TTS	5054	f5-tts-mlx	cloning¹	~0.54
fake	— (test tones, ships with Aviary)	5099	none	presets + cloning	—

¹ Cloning with Qwen3-TTS and F5-TTS requires the reference voice's transcript; the WebUI warns when it's missing.

Model weight licenses differ from this repo's MIT code license. MeloTTS and GPT-SoVITS are MIT throughout; MOSS-TTS-Nano and Qwen3-TTS weights are Apache-2.0; IndexTTS2 weights are non-commercial (written permission needed for commercial use); F5-TTS weights are CC-BY-NC; VoxCPM2 — see its model card. Check the upstream license before commercial use.

Requirements

• Apple Silicon Mac (tested with 128 GB unified memory; a few engines resident at once take 5–20 GB — the idle auto-unload keeps that in check)
• uv (manages every Python version and venv; you never touch pip)
• ffmpeg on PATH (brew install ffmpeg) — mp3 encode + upload normalizing
• Per-engine extras, installed once and only if you use that engine: MeloTTS wants brew install mecab; GPT-SoVITS wants brew install cmake (each engine's engine.toml lists its own [install] requirements; Aviary checks them before its first uv sync and prints the exact missing command)

Setup

git clone https://github.com/leoli-dev/aviary.git
cd aviary
cd hub && uv sync && cd ..     # the hub's own venv — the only manual install

That's the whole setup. Engine dependencies and model weights are not downloaded up front: the first time you start an engine, the hub creates its venv (uv sync), runs its one-time install steps, and the model downloads from Hugging Face on first load. Expect a few minutes per engine, once.

Start / stop the WebUI

./scripts/hub.sh start      # hub + WebUI on http://localhost:5050
./scripts/hub.sh status     # hub pid + engine states
./scripts/hub.sh logs       # tail the hub log
./scripts/hub.sh stop

Or run it in the foreground: cd hub && uv run aviary (Ctrl-C to stop). Stopping the hub stops all engine workers with it.

To keep it running across reboots, a launchd unit ships in deploy/com.aviary.hub.plist — install instructions are in the file's header comment.

The default config.toml binds 0.0.0.0, so other devices on your LAN (e.g. your iPhone) can open http://<mac-hostname>:5050 — the WebUI is responsive and installable as a PWA. Bind 127.0.0.1 instead if you want it local-only.

Securing the hub. Access control is off by default. Anyone who can reach the port can generate audio and change settings, so if the hub is exposed beyond a trusted LAN, set a token:

[hub]
auth_token = "a-long-random-string"   # generate: openssl rand -hex 24

Every /api/* call then needs Authorization: Bearer <token>, and the WebUI asks for it once per browser. Streams that cannot send a header (the SSE feed, <audio> playback, download links) accept ?token= instead — which puts the token in the hub's access log, on the same machine that already stores it in plaintext. The token is editable only in config.toml, never through the API it protects; the WebUI is never shown its value.

Using it

1. Engines page — start an engine (or don't: generating auto-starts it). Each card shows status, memory, RTF, live logs, and its configuration.
2. Studio — pick an engine; the voice options and parameter sliders come from the engine itself. Type text, Generate, listen, download. Toggle A/B compare to send the same text to several engines side by side.
3. Voices — upload 3–30 s of clean speech (with its transcript!) once, then clone that voice with any engine that supports cloning.
4. History — every generation kept: re-listen, re-download, or load it back into Studio.

API

Everything the WebUI does is plain HTTP.

Hub API (:5050) — engine lifecycle, generation jobs, voices, history, config. Interactive docs at http://localhost:5050/docs. The essentials:

# submit a generation (202 + job id; auto-starts the engine if stopped)
curl -X POST localhost:5050/api/generate -H 'content-type: application/json' \
  -d '{"engine": "voxcpm", "text": "Hello from Aviary.", "response_format": "wav"}'

# poll, then download
curl localhost:5050/api/jobs/<job_id>
curl -o out.wav "localhost:5050/api/jobs/<job_id>/audio?download=true"

Worker API (one per engine, on the ports in the table) — every worker speaks the same unified contract:

Endpoint	What it does
GET /	health — 2xx means the model is loaded
GET /capabilities	voice modes, parameter schema, languages, formats
POST /v1/audio/speech	OpenAI-style synthesis: JSON in, raw audio bytes out (X-Audio-Duration / X-Sample-Rate headers)
POST /clone	multipart cloning: text + reference_audio file + optional prompt_text transcript

# direct worker call, no hub involved
curl -X POST localhost:5077/v1/audio/speech -H 'content-type: application/json' \
  -d '{"input": "Direct to the engine.", "response_format": "wav"}' -o direct.wav

# cloning
curl -X POST localhost:5077/clone \
  -F text="Say this in the cloned voice." \
  -F reference_audio=@voice.wav -F prompt_text="transcript of voice.wav" -o clone.wav

OpenAPI documents: every running worker serves Swagger UI at http://localhost:<port>/docs and its spec at /openapi.json. Static copies for all engines are committed under docs/api/ — one <engine>.openapi.json each, including that engine's exact parameter schema as the /capabilities response example. Regenerate after changing an adapter:

uv run --project hub python scripts/export_openapi.py           # rewrite every engine's doc
uv run --project hub python scripts/export_openapi.py --check   # what CI enforces

Adapters import their model libraries lazily, so this needs no engine venv. A single worker can still be exported by hand from its own directory with uv run --no-sync python -m tts_hub_sdk.export_openapi adapter:MyEngine.

Engines with legacy clients also mount dialect routes — MOSS-TTS-Nano keeps the upstream multipart POST /api/generate (base64 JSON response), included in its OpenAPI doc.

Adding an engine

An engine is a folder under engines/ with three files. Copy engines/fake/ as a working template.

1. engine.toml — the manifest:

[engine]
name = "myengine"
title = "My Engine"
port = 5058
backend = "mlx"                 # informational label
python = "3.12"                 # uv provisions this
module = "adapter:MyEngine"     # class in adapter.py
autostart = false
idle_unload_min = 30            # 0 = stay resident

[install]                        # optional one-time steps on first start
brew = ["cmake"]                 # preflighted before the first uv sync
post = ["python -m something_download"]   # run inside the fresh venv

2. pyproject.toml — the engine's own dependencies plus the shared SDK:

[project]
name = "tts-engine-myengine"
requires-python = ">=3.12"
dependencies = ["some-tts-lib", "tts-hub-sdk"]

[tool.uv.sources]
tts-hub-sdk = { path = "../../sdk", editable = true }

3. adapter.py — one class:

class MyEngine:
    capabilities = {
        "engine": "myengine",
        "voice_modes": ["preset_list", "reference_audio"],  # what the UI offers
        "voices": [{"id": "alto", "label": "Alto"}],
        "languages": ["en"],
        "params": [   # becomes sliders/selects in Studio, coerced by the SDK
            {"name": "speed", "type": "float", "min": 0.5, "max": 2.0,
             "default": 1.0, "step": 0.05, "label": "Speed"},
        ],
        "formats": ["wav", "mp3"],
        "sample_rate": 24000,
        "max_chunk_chars": 400,   # hub splits longer text; 0 = engine handles it
    }

    def load(self) -> None:
        ...  # blocking model load; runs once on the inference thread

    def synthesize(self, text, params, ref_audio=None, ref_text=None):
        ...  # return (waveform_ndarray, sample_rate)

Then click Settings → Rescan engines (or restart the hub). The first Start builds the venv, runs install.post, downloads weights — and the WebUI needs zero changes: forms render from capabilities.

Removing an engine: stop it, delete (or move away) its folder, rescan. Hiding without deleting: Disable engine in its configuration panel.

Configuration (two layers)

Engine defaults live in each engine's engines/<id>/engine.toml (shown above). Your overrides live in config.toml at the repo root, written by the Settings page or edited by hand:

[hub]
host = "0.0.0.0"        # 127.0.0.1 = local only
port = 5050
max_upload_mb = 100

[engines.melotts]
port = 5066             # override the manifest default
autostart = true
idle_unload_min = 0     # keep resident
enabled = false         # hide from Studio without deleting files
env = { MELO_LANG = "ZH" }   # worker env vars — model variants go here

Precedence: config.toml > engine.toml. Overrides for a running engine apply on its next start. Model choice is set via env vars read by each adapter (VOXCPM_MODEL, QWEN3_TTS_MODEL, INDEXTTS_MODEL, F5_MODEL, MOSS_MODEL, MELO_LANG — see each adapter.py header).

Repo layout

hub/        management plane: process supervision, SSE, jobs, SQLite, WebUI (Py3.12)
sdk/        tts_hub_sdk — the shared worker layer: unified HTTP contract,
            lenient param coercion, wav/mp3 encoding, single-thread
            inference executor, orphan self-exit, OpenAPI export
engines/    one folder per engine (manifest + pyproject + adapter)
docs/api/   static OpenAPI documents, one per engine
scripts/    hub.sh start/stop/status/logs
deploy/     launchd unit for the hub
config.toml hub settings + per-engine overrides
data/       runtime state (gitignored): audio, voices, logs, hub.db

Acknowledgements

Aviary stands on the engines' upstream teams and the Apple Silicon runtimes that host them: mlx-audio, mlx-indextts, f5-tts-mlx, and GPT-SoVITS's built-in API server.