Call Analyzer

Self-hosted speech analytics for call recordings: transcription (Russian-first, multi-language), speaker diarization, and semantic search over transcripts. Audio stays on your machine.

• Transcription — OpenAI Whisper, batch uploads, progress tracking per file
• Diarization — pyannote.audio splits the conversation by speaker; known speakers are recognized across calls via voice embeddings
• Semantic search — FAISS index over multilingual sentence embeddings, search transcripts by meaning with timestamps
• Web UI — Next.js 15 / React 19 with audio playback synced to the transcript

Requirements

• Docker and Docker Compose
• HuggingFace token with access to the pyannote/speaker-diarization models (needed for diarization only)
• NVIDIA GPU recommended; CPU works but transcription is slow
• 8 GB RAM minimum, 16 GB recommended; ~10 GB disk for models and data

Quick start

git clone https://github.com/liqlos/call-analyzer.git
cd call-analyzer
cp .env.example .env

Edit .env and set HUGGINGFACE_TOKEN. Then:

docker compose up -d

The first run downloads Whisper and pyannote models (several GB). When it's up:

• Web UI: http://localhost:3000
• API docs (Swagger): http://localhost:8000/docs
• Health check: http://localhost:8000/health

Upload audio on the Upload page. Once a call is transcribed, it can be played back with its transcript and searched.

Configuration

Environment variables for .env (used by docker compose):

Variable	Description	Default
DEVICE	Inference device: cuda, cpu, or empty for auto-detect (falls back to cpu when CUDA is unavailable)	auto
WHISPER_MODEL	Whisper model size: tiny, base, small, medium, large	tiny
HUGGINGFACE_TOKEN	HuggingFace token for pyannote models	—
DIARIZATION_ENABLED	Set to false to skip diarization	true
AUTO_REBUILD_INDEX	Rebuild the FAISS index on startup	false
LOG_LEVEL	Uvicorn log level	INFO
NEXT_PUBLIC_API_URL	Backend URL as seen from the browser	http://localhost:8000

Defaults shown are from .env.example / docker-compose.yml.

The Whisper model can also be changed at runtime on the Settings page or via PUT /api/config — but only while no data is indexed.

API

Interactive docs live at /docs. All endpoints accept and return JSON except file uploads and audio streaming.

Audio processing

Method	Endpoint	Description
POST	/api/audio/upload/batch	Upload multiple files for transcription
POST	/api/audio/upload	Upload a single file
GET	/api/audio/tasks/{task_id}	Task status and per-file progress (?include_files=true)
GET	/api/audio/tasks	List tasks (limit, offset, include_files)
GET	/audio/{file_name}	Stream a processed audio file (supports Range requests)
POST	/api/audio/tasks/resume	Resume interrupted tasks after a restart
DELETE	/api/audio/tasks/clear	Delete tasks older than N days (?older_than_days=7)

Search

Method	Endpoint	Description
GET	/api/search?q=...&limit=10	Semantic search over transcripts
POST	/api/rebuild-index	Rebuild the FAISS index from stored transcripts

Speakers

Method	Endpoint	Description
GET / POST	/api/audio/speakers	List / create speakers
GET / PUT / DELETE	/api/audio/speakers/{speaker_id}	Speaker details, rename, delete
POST	/api/audio/speakers/{speaker_id}/embedding	Set a speaker's voice embedding directly
POST	/api/audio/speakers/recompute	Re-match speakers across all calls

Service and settings

Method	Endpoint	Description
GET / HEAD	/health	Health check
GET	/api/status	Detailed system status
GET / PUT	/api/config	Read / update settings (token, model, language)
POST	/api/config/reset	Reset settings to defaults
POST	/api/clean-data	Delete all uploads, transcripts, and indices

Example — batch upload:

curl -X POST http://localhost:8000/api/audio/upload/batch \
  -F "files=@call1.mp3" \
  -F "files=@call2.wav" \
  -F "language=ru"

Response:

{
  "task_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "pending",
  "message": "Files uploaded successfully",
  "files": []
}

Poll GET /api/audio/tasks/{task_id}?include_files=true for progress.

Development

Backend (Python 3.10+):

python -m venv venv && source venv/bin/activate
pip install -r backend/requirements.txt -r backend/requirements-test.txt
uvicorn main:app --reload --app-dir backend

Frontend (Node 18+): see frontend/README.md.

Utility scripts:

python scripts/check_health.py                    # verify the running instance
python scripts/test_search.py --query "query"     # try the search endpoint
./scripts/init_env.sh                             # create local data directories

Troubleshooting

• 401 when downloading models — some pyannote models require accepting their license on the model page before your token can fetch them. Open the model page on huggingface.co, agree to the terms, retry.
• Slow transcription — use a smaller WHISPER_MODEL, or run on a machine with an NVIDIA GPU (DEVICE=cuda). The first request is slower because models load on demand.
• Diarization fails or speakers are missing — check HUGGINGFACE_TOKEN and license acceptance above; otherwise set DIARIZATION_ENABLED=false to transcribe without speaker labels.
• Out of memory — large Whisper models need several GB of RAM/VRAM; base or small is enough for most call recordings.
• Lost data after restart — data lives in the call_analyzer_data Docker volume; don't run docker compose down -v unless you want to wipe it.

Project structure

├── backend/             # FastAPI service: transcription, diarization, search
│   ├── api/             #   routers
│   └── main.py          #   app entrypoint
├── frontend/            # Next.js UI: upload, playback, search, speakers
├── scripts/             # utility scripts (health check, search test)
└── docker-compose.yml

License

MIT

Author

Aleksei Bukhalov

• Website: https://bukhalov.com
• GitHub: https://github.com/liqlos