BharatVoiceAI

Open-source, production-grade AI services for Indian languages.
ASR, Machine Translation, Text-to-Speech, Language Identification, and NLU — covering 22 Indic languages.

---

Why this exists

India has 870M+ internet users accessing content in Indic languages, but almost all AI infrastructure is English-first. This project provides:

• A unified REST/WebSocket API for ASR, MT, TTS, LID, and NLU across 22 Indian languages
• Swappable backends — use BHASHINI, AI4Bharat, Sarvam AI, or your own fine-tuned models
• Docker, Kubernetes, CI/CD, monitoring, rate-limiting, and auth out of the box
• DPDP 2023 compliant — audio never logged by default, consent flows included
• MIT license, no CLA required

---

Architecture

Client Applications (Mobile / WhatsApp / IVR / Web)
        |
        v  REST / WebSocket
API Gateway (FastAPI)
  Auth - Rate Limiting - Routing - Observability
        |
  +-----+-----+-----+-----+-----+
  |     |     |     |     |     |
 ASR   MT   TTS   LID   NLU
  |     |     |     |     |
  +-----+-----+-----+-----+
        |
  Model Backend Layer
  (BHASHINI / AI4Bharat / Sarvam AI / Local ONNX)

---

Services

Service	What it does	Primary backend	Status
ASR	Speech to Text	Whisper large-v3	Ready
MT	Machine Translation	IndicTrans2 (1B)	Ready
TTS	Text to Speech	Indic Parler-TTS	Ready
LID	Language Identification	Hybrid (XLM-R + IndicLID-FTR)	Ready
NLU	Intent + Entity extraction	Rule engine + transliteration	Ready

Streaming ASR (WebSocket) and OCR are in progress.

Note: This repo has been tested end-to-end with local models only (Whisper v3, IndicTrans2, Indic Parler-TTS, IndicLID-FTR, gTTS). The BHASHINI and Sarvam AI backends are implemented but have not been tested with live API keys. If you have BHASHINI/Sarvam credentials, set them in .env and the backends should work — but expect rough edges.

---

Quick start

Local Python setup (recommended)

# 1. Create virtual environment (Python 3.11+)
python3 -m venv .venv
source .venv/bin/activate

# 2. Install base + dev dependencies
pip install -e ".[dev]"

# 3. Install AI model backends (requires ~8GB disk, downloads models on first use)
pip install -e ".[models]"

# 4. Copy and configure environment
cp .env.example .env
# Edit .env — at minimum, the defaults work for local testing

# 5. HuggingFace setup (required for gated models)
#
# a) Create a free account at https://huggingface.co/join
# b) Create an access token at https://huggingface.co/settings/tokens
# c) Accept the license for these gated models (click "Agree" on each page):
#    - https://huggingface.co/ai4bharat/indic-parler-tts
#    - https://huggingface.co/ai4bharat/IndicLID-FTR
# d) Login on your machine:
huggingface-cli login

# 6. Install system dependency (for audio format conversion)
sudo apt install ffmpeg   # Ubuntu/Debian
# brew install ffmpeg      # macOS

# 7. Run the API (no Docker, no database needed)
uvicorn api.main:app --reload --port 8000

# Optional: if you want rate limiting persistence or admin features,
# start PostgreSQL + Redis via Docker:
# docker compose up -d postgres redis
# and set VAI_DB_URL and VAI_REDIS_URL in .env

Interactive docs at http://localhost:8000/docs.

Models download automatically on first request (~3-8GB per model, cached in /tmp/vai_models). If you don't have a GPU, set DEVICE=cpu in .env — inference will be slower but works.

Frontend

The frontend is a single HTML file — no build step or Node.js needed.

cd frontend
python3 -m http.server 5500
# Open http://localhost:5500

Or just open frontend/index.html directly in your browser.

Changing ports:

• Backend on a different port (e.g. 9000): start with uvicorn api.main:app --port 9000, then update the "API Base" field in the top-right corner of the frontend to http://localhost:9000
• Frontend on a different port (e.g. 3000): run python3 -m http.server 3000 and open http://localhost:3000
• Backend on a remote server: set the "API Base" field in the frontend to http://your-server-ip:8000

Docker Compose (full stack)

Docker Compose runs the API with PostgreSQL, Redis, Prometheus, and Grafana. Use this when you want the full production-like setup with rate limiting, monitoring, and database.

Note: AI models are downloaded inside the container on first request. You must set HF_TOKEN in your environment for gated models, and accept model licenses on HuggingFace first (see step 5 above).

cp .env.example .env
# Set VAI_DB_URL=postgresql+asyncpg://vai:vai@postgres:5432/vai in .env
# Set HF_TOKEN=your_huggingface_token in .env (for gated models)
docker compose up -d
curl http://localhost:8000/health

---

API examples

Speech-to-Text

curl -X POST http://localhost:8000/v1/asr \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "audio_url": "https://example.com/audio.wav",
    "lang": "hi",
    "consent_given": true
  }'

Translate

curl -X POST http://localhost:8000/v1/translate \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello, how are you?", "src_lang": "en", "tgt_lang": "hi"}'

Text-to-Speech

curl -X POST http://localhost:8000/v1/tts \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "नमस्ते", "lang": "hi", "voice": "female_1", "format": "wav"}'

Language Identification

curl -X POST http://localhost:8000/v1/lid \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "mera naam Rahul hai"}'

Intent / NLU

curl -X POST http://localhost:8000/v1/nlu \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "मेरा बिल दिखाओ", "lang": "hi", "domain": "billing"}'

---

Supported languages

Hindi, Tamil, Telugu, Bengali, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia, Assamese, Urdu, Sanskrit, Maithili, Konkani, Manipuri, Bodo, Dogri, Kashmiri, Nepali, Sindhi, Santali — plus English.

Coverage varies by service. See the docs for per-language availability.

---

Project structure

bharatvoiceai/
  api/                  FastAPI application
    main.py             App entry point
    routers/            Route handlers (asr, mt, tts, lid, nlu)
    middleware/          Auth, rate limiting, logging
    schemas/            Pydantic request/response models
  services/             Core service implementations
    asr/                Speech-to-Text backends
    mt/                 Machine Translation backends
    tts/                Text-to-Speech backends
    lid/                Language Identification backends
    nlu/                Intent + entity extraction
  core/                 Config, logging, database, metrics
  frontend/             Browser-based demo UI
  infra/                Docker, Kubernetes, monitoring
  tests/                Unit, integration, load tests
  scripts/              Evaluation and setup scripts

---

Configuration

All configuration is via environment variables. Copy .env.example to .env and edit.

Key variables:

VAI_ASR_BACKEND=indicwhisper     # indicwhisper | bhashini | wav2vec2
VAI_MT_BACKEND=indictrans2       # indictrans2 | bhashini | sarvam
VAI_TTS_BACKEND=indic_parler     # indic_parler | bhashini
VAI_LID_BACKEND=smart            # smart | indiclid | bhashini
VAI_NLU_BACKEND=rule+llm         # rule | llm | rule+llm

BHASHINI_USER_ID=                # free at bhashini.gov.in
BHASHINI_API_KEY=
SARVAM_API_KEY=                  # for LLM-based NLU

VAI_JWT_SECRET=...               # min 32 chars
STATIC_API_KEYS=your-key-here    # comma-separated

---

Integrating with your system

BharatVoiceAI is a standalone service. You call it over HTTP from your existing app — no tight coupling needed.

As a microservice

Point your backend to the BharatVoiceAI endpoints:

# From your Django/Flask/Node app
import requests

response = requests.post("http://bharatvoiceai-host:8000/v1/translate", json={
    "text": user_input,
    "src_lang": "hi",
    "tgt_lang": "en",
}, headers={"X-API-Key": "your-key"})

translation = response.json()["translation"]

Works the same from any language — JavaScript, Go, Java, curl. It's just REST.

About the database

The database is optional. It's used for rate limiting state and admin features — not for storing user data or audio. All AI services (ASR, MT, TTS, LID, NLU) work fully without any database.

Run without any database — just leave VAI_DB_URL empty in .env:

VAI_DB_URL=

The API starts normally, skips DB init, and rate limiting falls back to in-memory counters.

Use a lightweight file-based DB (no Postgres install needed):

VAI_DB_URL=sqlite+aiosqlite:///./vai.db

Use your own PostgreSQL:

VAI_DB_URL=postgresql+asyncpg://user:pass@your-db-host:5432/your_db

psql -f scripts/init_db.sql -h your-db-host -U user -d your_db

Use MySQL/MariaDB:

pip install aiomysql
VAI_DB_URL=mysql+aiomysql://user:pass@your-db-host:3306/your_db

Redis is also optional. It's used for rate limiting counters and response caching. Without it, rate limiting uses in-memory storage and caching is disabled. Everything else works.

---

Testing

pytest tests/unit/ -v
pytest tests/integration/ -v
python scripts/eval_asr.py --dataset iitm_hindi --backend indicwhisper
python scripts/eval_mt.py --dataset flores200 --src hi --tgt en

---

Contributing

See CONTRIBUTING.md. PRs welcome — just sign off your commits with Signed-off-by:.

---

License

MIT. See LICENSE.

Third-party models have their own licenses:

• IndicTrans2, IndicWhisper: MIT
• Indic Parler-TTS: Apache 2.0
• BHASHINI APIs: Government of India ToU
• Sarvam models: Apache 2.0

---

Acknowledgements

Built on work by AI4Bharat / IIT Madras, BHASHINI / MeitY, Sarvam AI, and the EkStep Foundation.