Replace ONNX Runtime with Kitten Inference Engine

.gitignore

@@ -29,6 +29,7 @@ Temporary Items
 .venv*
 
 *.wav
+summary.json
 
 # Byte-compiled / optimized / DLL files
 __pycache__/
@@ -245,4 +246,4 @@ marimo/_lsp/
 __marimo__/
 
 # Streamlit
-.streamlit/secrets.toml
+.streamlit/secrets.toml

MANIFEST.in

@@ -1,9 +1,9 @@
 include README.md
 include LICENSE
 include requirements.txt
+recursive-include docs *.md
 recursive-include kittentts *.py
 recursive-include kittentts *.json
 recursive-include kittentts *.txt
-recursive-include kittentts *.onnx
 global-exclude __pycache__
 global-exclude *.py[co]

README.md

@@ -11,9 +11,9 @@
   <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-green.svg" alt="License"></a>
 </p>
 
-> **New:** Kitten TTS v0.8 is out -- 15M, 40M, and 80M parameter models now available.
+> **New:** Kitten TTS v0.8.2 uses the Kitten Inference Engine (`kitten-inference`).
 
-Kitten TTS is an open-source, lightweight text-to-speech library built on ONNX. With models ranging from 15M to 80M parameters (25-80 MB on disk), it delivers high-quality voice synthesis on CPU without requiring a GPU.
+Kitten TTS is an open-source, lightweight text-to-speech library built on the Kitten Inference Engine. With models ranging from 15M to 80M parameters, it delivers high-quality voice synthesis on CPU wheels for Linux, Windows, macOS, and Android/Termux.
 
 > **Status:** Developer preview -- APIs may change between releases.
 
@@ -23,6 +23,7 @@ Kitten TTS is an open-source, lightweight text-to-speech library built on ONNX.
 
 - [Features](#features)
 - [Available Models](#available-models)
+- [Benchmarks](#benchmarks)
 - [Demo](#demo)
 - [Quick Start](#quick-start)
 - [API Reference](#api-reference)
@@ -35,22 +36,51 @@ Kitten TTS is an open-source, lightweight text-to-speech library built on ONNX.
 ## Features
 
 - **Ultra-lightweight** -- Model sizes from 25 MB (int8) to 80 MB, suitable for edge deployment
-- **CPU-optimized** -- ONNX-based inference runs efficiently without a GPU
+- **Kitten Inference Engine** -- C++ backend for Linux, Windows, macOS, and Android CPU runtimes, with Metal available on Apple Silicon
 - **8 built-in voices** -- Bella, Jasper, Luna, Bruno, Rosie, Hugo, Kiki, and Leo
-- **Adjustable speech speed** -- Control playback rate via the `speed` parameter
+- **Stable Python API** -- The existing `KittenTTS(...).generate(...)` API remains in place
 - **Text preprocessing** -- Built-in pipeline handles numbers, currencies, units, and more
 - **24 kHz output** -- High-quality audio at a standard sample rate
 
 ## Available Models
 
 | Model | Parameters | Size | Download |
 |---|---|---|---|
-| kitten-tts-mini | 80M | 80 MB | [KittenML/kitten-tts-mini-0.8](https://huggingface.co/KittenML/kitten-tts-mini-0.8) |
-| kitten-tts-micro | 40M | 41 MB | [KittenML/kitten-tts-micro-0.8](https://huggingface.co/KittenML/kitten-tts-micro-0.8) |
-| kitten-tts-nano | 15M | 56 MB | [KittenML/kitten-tts-nano-0.8](https://huggingface.co/KittenML/kitten-tts-nano-0.8-fp32) |
-| kitten-tts-nano (int8) | 15M | 25 MB | [KittenML/kitten-tts-nano-0.8-int8](https://huggingface.co/KittenML/kitten-tts-nano-0.8-int8) |
+| kitten-tts-mini | 80M | native weights | `KittenML/kitten-tts-mini-0.8` via [KittenML/meownn-models](https://huggingface.co/KittenML/meownn-models) |
+| kitten-tts-mini (int8) | 80M | native weights | `KittenML/kitten-tts-mini-0.8-int8` via [KittenML/meownn-models](https://huggingface.co/KittenML/meownn-models) |
+| kitten-tts-micro | 40M | native weights | `KittenML/kitten-tts-micro-0.8` via [KittenML/meownn-models](https://huggingface.co/KittenML/meownn-models) |
+| kitten-tts-micro (int8) | 40M | native weights | `KittenML/kitten-tts-micro-0.8-int8` via [KittenML/meownn-models](https://huggingface.co/KittenML/meownn-models) |
+| kitten-tts-nano | 15M | native weights | `KittenML/kitten-tts-nano-0.1` via [KittenML/meownn-models](https://huggingface.co/KittenML/meownn-models) |
+| kitten-tts-nano (int8) | 15M | native weights | `KittenML/kitten-tts-nano-0.8-int8` via [KittenML/meownn-models](https://huggingface.co/KittenML/meownn-models) |
+
+Built-in model aliases use packaged arch JSONs and download W1X1 weights plus
+voice styles from the native weights repository. Custom native model
+repositories may also publish `CPP1` configs with `arch_file`, `weights_file`,
+and voice file entries in `config.json`.
+For the 15M FP32 model, `KittenML/kitten-tts-nano-0.8` and
+`KittenML/kitten-tts-nano-0.8-fp32` are accepted aliases for the same native
+weights.
+
+## Benchmarks
+
+CPU-only Kitten Inference Engine was benchmarked against ONNX Runtime with the
+80M int8 model. Text length = 18 words / 88 characters.
+Lower is better.
 
-> **Note:** Some users have reported issues with the `kitten-tts-nano-0.8-int8` model. If you encounter problems, please [open an issue](https://github.com/KittenML/KittenTTS/issues).
+<p align="center">
+  <img src="docs/assets/benchmarks/cpu_onnx_summary.svg" alt="CPU latency and peak memory vs ONNX on Android, Linux, and macOS">
+</p>
+
+| Host | Latency vs ONNX | Peak RSS vs ONNX |
+|---|---:|---:|
+| Android ARM64 / Cortex-A77 | 2.9x faster at `t=1`; 2.2x at `t=8` | 1.9x lower |
+| Linux x86_64 / Core i7-12650H | 1.1x faster at `t=1`; 1.2x at `t=8` | 2.5x lower |
+| macOS ARM64 / Apple Silicon | 5.4x faster at `t=1`; 2.9x at `t=8` | 3.0x lower |
+
+`t=1` and `t=8` are CPU host-thread settings. The best setting varies by CPU;
+both are shown for transparency. Data is from June 2026 benchmark runs; ONNX
+uses the latest run with model rows when a newer run did not emit ONNX rows. No
+CUDA or Metal backend results are included.
 
 ## Demo
 
@@ -62,23 +92,65 @@ Try Kitten TTS directly in your browser on [Hugging Face Spaces](https://hugging
 
 ## Quick Start
 
-### Prerequisites
+### Installation
 
-- Python 3.8 or later
-- pip
+KittenTTS is a pure Python wheel for CPython 3.8+. It depends on
+`kitten-inference==0.1.1`, which is published as platform-specific native
+wheels. Pip selects the native wheel that matches your Python version, OS, and
+CPU.
 
-### Installation
+```bash
+pip install https://github.com/KittenML/KittenTTS/releases/download/0.8.2/kittentts-0.8.2-py3-none-any.whl
+```
+
+If pip reports that no `kitten-inference` distribution is available, use one of
+the Python/platform combinations listed below or build the native engine wheel
+from source.
+
+Android / Termux currently requires the CPython 3.13 ARM64 wheel:
+
+```bash
+pkg install -y python espeak libsndfile
+python --version
+python -m pip install --upgrade pip
+python -m pip install https://github.com/KittenML/KittenTTS/releases/download/0.8.2/kittentts-0.8.2-py3-none-any.whl
+```
+
+If Termux reports a Python version other than 3.13, pip will not select the
+current Android wheel.
+
+For local release testing, put `kittentts-0.8.2-py3-none-any.whl` in Android
+Downloads and replace the URL with:
 
 ```bash
-pip install https://github.com/KittenML/KittenTTS/releases/download/0.8.1/kittentts-0.8.1-py3-none-any.whl
+termux-setup-storage
+python -m pip install ~/storage/downloads/kittentts-0.8.2-py3-none-any.whl
 ```
 
+| Target | Python | Engine wheel |
+|---|---|---|
+| Linux x86_64 CPU | CPython 3.8-3.14 | `kitten_inference-*-cp3*-cp3*-manylinux_*_x86_64.whl` |
+| Linux ARM64 / aarch64 CPU | CPython 3.8-3.14 | `kitten_inference-*-cp3*-cp3*-manylinux_*_aarch64.whl` |
+| Windows x86_64 CPU | CPython 3.8-3.14 | `kitten_inference-*-cp3*-cp3*-win_amd64.whl` |
+| Windows ARM64 CPU | CPython 3.11-3.14 | `kitten_inference-*-cp3*-cp3*-win_arm64.whl` |
+| macOS ARM64 / Apple Silicon CPU+Metal | CPython 3.8-3.14 | `kitten_inference-*-cp3*-cp3*-macosx_11_0_arm64.whl` |
+| macOS x86_64 / Intel CPU | CPython 3.8-3.14 | `kitten_inference-*-cp3*-cp3*-macosx_11_0_x86_64.whl` |
+| Android ARM64 / Termux | CPython 3.13 experimental | `kitten_inference-*-cp313-cp313-android_*_arm64_v8a.whl` |
+
+Support notes:
+
+- The native wheel set covers only the targets listed above.
+- Not covered by the current wheels: Windows x86 32-bit, Linux x86 32-bit, Linux musl/Alpine, CUDA, and unusual Python/platform tags not present on the `kitten-inference` release.
+- ARM64 wheels are built for modern ARMv8.2 dot-product-capable cores. Older ARM devices without dot-product support have not been validated and may break.
+
 ### Basic Usage
 
 ```python
 from kittentts import KittenTTS
 
 model = KittenTTS("KittenML/kitten-tts-mini-0.8")
+# Smaller/faster option:
+# model = KittenTTS("KittenML/kitten-tts-nano-0.1")
 audio = model.generate("This high-quality TTS model runs without a GPU.", voice="Jasper")
 
 import soundfile as sf
@@ -88,39 +160,32 @@ sf.write("output.wav", audio, 24000)
 ### Advanced Usage
 
 ```python
-# Adjust speech speed (default: 1.0)
+# speed > 1.0 speaks faster; speed < 1.0 speaks slower.
 audio = model.generate("Hello, world.", voice="Luna", speed=1.2)
 
 # Save directly to a file
-model.generate_to_file("Hello, world.", "output.wav", voice="Bruno", speed=0.9)
+model.generate_to_file("Hello, world.", "output.wav", voice="Bruno", speed=1.0)
 
 # List available voices
 print(model.available_voices)
 # ['Bella', 'Jasper', 'Luna', 'Bruno', 'Rosie', 'Hugo', 'Kiki', 'Leo']
-```
-
-### Using with GPU
 
+# Apple Silicon Metal
+metal_model = KittenTTS("KittenML/kitten-tts-mini-0.8", backend="metal")
 ```
-pip install -r requirements_gpu.txt
-```
-
-```python
-m = KittenTTS("KittenML/kitten-tts-mini-0.8", backend="cuda")
-```
-
-Check out `example_cuda.py` 
 
 ## API Reference
 
-### `KittenTTS(model_name, cache_dir=None)`
+### `KittenTTS(model_name, cache_dir=None, backend=None)`
 
-Load a model from Hugging Face Hub.
+Load a model from Hugging Face Hub or a local native model directory. `backend`
+is optional; omit it for CPU, or pass `"metal"` on Apple Silicon.
 
 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `model_name` | `str` | `"KittenML/kitten-tts-nano-0.8"` | Hugging Face repository ID |
+| `model_name` | `str` | `"KittenML/kitten-tts-nano-0.1"` | Hugging Face repository ID or local native model directory |
 | `cache_dir` | `str` | `None` | Local directory for caching downloaded model files |
+| `backend` | `str` | `None` | `"cpu"` or `"metal"` on Apple Silicon; defaults to CPU |
 
 ### `model.generate(text, voice, speed, clean_text)`
 
@@ -129,8 +194,8 @@ Synthesize speech from text, returning a NumPy array of audio samples at 24 kHz.
 | Parameter | Type | Default | Description |
 |---|---|---|---|
 | `text` | `str` | -- | Input text to synthesize |
-| `voice` | `str` | `"expr-voice-5-m"` | Voice name (see available voices) |
-| `speed` | `float` | `1.0` | Speech speed multiplier |
+| `voice` | `str` | `"expr-voice-5-m"` | Voice alias such as `"Bella"` or native voice ID |
+| `speed` | `float` | `1.0` | Speech speed multiplier; values above `1.0` are faster and values below `1.0` are slower |
 | `clean_text` | `bool` | `False` | Preprocess text (expand numbers, currencies, etc.) |
 
 ### `model.generate_to_file(text, output_path, voice, speed, sample_rate, clean_text)`
@@ -141,8 +206,8 @@ Synthesize speech and write directly to an audio file.
 |---|---|---|---|
 | `text` | `str` | -- | Input text to synthesize |
 | `output_path` | `str` | -- | Path to save the audio file |
-| `voice` | `str` | `"expr-voice-5-m"` | Voice name |
-| `speed` | `float` | `1.0` | Speech speed multiplier |
+| `voice` | `str` | `"expr-voice-5-m"` | Voice alias such as `"Bella"` or native voice ID |
+| `speed` | `float` | `1.0` | Speech speed multiplier; values above `1.0` are faster and values below `1.0` are slower |
 | `sample_rate` | `int` | `24000` | Audio sample rate in Hz |
 | `clean_text` | `bool` | `True` | Preprocess text (expand numbers, currencies, etc.) |
 
@@ -165,20 +230,28 @@ When `return_spans=True`, the result includes original-to-normalized character s
 
 ### `model.available_voices`
 
-Returns a list of available voice names: `['Bella', 'Jasper', 'Luna', 'Bruno', 'Rosie', 'Hugo', 'Kiki', 'Leo']`
+For built-in models, returns the friendly voice aliases:
+`['Bella', 'Jasper', 'Luna', 'Bruno', 'Rosie', 'Hugo', 'Kiki', 'Leo']`.
+Native voice IDs such as `"expr-voice-5-m"` are also accepted.
 
 ## System Requirements
 
-- **Operating system:** Linux, macOS, or Windows
-- **Python:** 3.8 or later
-- **Hardware:** Runs on CPU; no GPU required
-- **Disk space:** 25-80 MB depending on model variant
+- **Operating system:** Linux x86_64, Linux ARM64, Windows x86_64, Windows ARM64, macOS x86_64, macOS ARM64/Apple Silicon, or Android ARM64/Termux for the current wheels
+- **Python:** CPython 3.8+ for KittenTTS; the native `kitten-inference` wheel must match your Python version and platform
+- **Hardware:** CPU by default; Metal on Apple Silicon with `backend="metal"` is covered by smoke tests on macOS ARM64 wheels, but full Metal performance/quality benchmarking is still limited
+- **Disk space:** Depends on the native model variant and weights bundle
+
+The current wheel set is not a universal hardware claim. Older x86 CPUs, older
+ARM devices without ARMv8.2 dot-product support, musl/Alpine Linux, 32-bit
+platforms, and platform tags outside the published wheel matrix should be
+treated as unvalidated unless a separate compatible wheel and smoke test are
+provided.
 
 A virtual environment (conda, venv, or similar) is recommended to avoid dependency conflicts.
 
 ## Roadmap
 
-- [ ] Release optimized inference engine
+- [x] Release optimized inference engine
 - [ ] Release mobile SDK
 - [ ] Release higher quality TTS models
 - [ ] Release multilingual TTS