A FastAPI service that turns long audio into a structured summary, running entirely on local models. Built around the problem that makes AI features awkward to ship: the work takes minutes, but an HTTP request can't.
Submit a source, get a job id back immediately, poll it for the result.
Transcribing an hour of audio and summarizing it takes several minutes of CPU work. A synchronous endpoint that does this holds a connection open the whole time, times out behind most proxies and load balancers, blocks a worker, and gives the caller nothing to show until it either finishes or dies.
So the API is built as a job queue over HTTP:
sequenceDiagram
participant C as Client
participant A as API
participant W as Worker thread
C->>A: POST /summarize
A->>A: validate, create job
A->>W: schedule (background)
A-->>C: 202 {id, status: "pending"}
W->>W: download → transcribe → summarize
C->>A: GET /jobs/{id}
A-->>C: {status: "running"}
C->>A: GET /jobs/{id}
A-->>C: {status: "done", transcription, summary}
The submit endpoints return in milliseconds. The pipeline is synchronous,
CPU-bound Python, so it is declared as a plain def and FastAPI runs it in a
worker thread — the event loop stays free to serve polls while the job runs.
| Decision | Why |
|---|---|
| 202 + polling instead of a blocking call | A minutes-long request is a timeout waiting to happen. The client gets an id it can poll, retry, or abandon. |
Dependencies built once in lifespan |
The Whisper model costs seconds and GBs of RAM to load. Loading it per request would dominate the runtime; here it's warm before the first request lands. |
| Map-reduce summarization | A 65k-character transcript doesn't fit a context window. Above a threshold the text is chunked, each chunk summarized, then the partials merged — so long inputs degrade gracefully instead of being silently truncated. |
| Per-job temporary directory | Audio is an intermediate artifact. A 1-hour video produces a ~100 MB MP3; a shared directory would grow without bound, leak .part files from failed downloads, and let two jobs on the same video overwrite each other. |
| Generic error to the client, traceback to the logs | Exceptions carry local filesystem paths. The caller gets "Processing failed."; the operator gets the stack trace. |
| Streaming upload with a size cap | The cap is enforced while writing, not read from Content-Length — a header the client controls and can lie about. |
| Injected config and collaborators | Settings is passed in, not read from globals, so every component can be tested against fakes. The test suite runs with no network, no Whisper and no LLM. |
FastAPI · uv (lockfile-based dependency management) · Whisper (local transcription) · Ollama (local LLM via its OpenAI-compatible API) · yt-dlp · pytest · Docker Compose · Streamlit (optional web client)
Nothing is sent to a third-party API — both models run on the machine.
ffmpeg is required by yt-dlp and Whisper, and is not a Python package:
brew install ffmpeg
uv syncYou also need Ollama running with the model pulled:
ollama pull qwen3:8bThen:
uv run uvicorn main:app --reloadInteractive docs at http://localhost:8000/docs
An optional Streamlit UI for trying the service by hand — submit a URL or a file, watch the job status, read the summary. It is a pure HTTP client: it talks only to the API, so it stays honest to the same contract as any other caller and is never needed to run or deploy the service.
uv sync --group ui # one-time: install the UI extras
uv run streamlit run streamlit_app.py # with the API already runningIt defaults to http://localhost:8000; point it elsewhere from the sidebar or
with SUMMARIZER_API_URL.
docker compose up --build
docker compose exec ollama ollama pull qwen3:8bThis brings up the API and an Ollama container together. On macOS, pointing the API at a native Ollama instead is significantly faster — Linux containers get no access to the Apple Silicon GPU, so a containerized LLM is CPU-only:
OLLAMA_BASE_URL=http://host.docker.internal:11434/v1 docker compose up apiConfiguration is overridden through environment variables or a .env file —
see .env.example for the full list and defaults.
| Method | Route | Description |
|---|---|---|
GET |
/health |
Liveness probe and the active model configuration |
POST |
/summarize |
Queue a video URL → 202 with the job id |
POST |
/summarize/upload |
Queue an uploaded audio file → 202 with the job id |
GET |
/jobs/{id} |
Job status and, once ready, transcription and summary |
curl -X POST localhost:8000/summarize \
-H 'content-type: application/json' \
-d '{"url": "https://www.youtube.com/watch?v=BEtBxcem6-M"}'
# -> 202 {"id": "…", "status": "pending", "source": "url", …}
curl -X POST localhost:8000/summarize/upload -F 'file=@talk.mp3'
# -> 202 {"id": "…", "status": "pending", "source": "upload", …}
curl localhost:8000/jobs/<id>
# -> status: pending -> running -> done | errorBoth sources feed the same pipeline; the upload path simply skips the download stage.
app/
config.py Settings (pydantic-settings), read from env or .env
prompts.py System prompts and summary templates
download.py Downloader — media URL to MP3 via yt-dlp
transcribe.py Transcriber — local Whisper, model loaded once
summarize.py Summarizer — LLM via OpenAI-compatible API, map-reduce
pipeline.py VideoSummarizer — orchestrates the stages
main.py FastAPI app: lifespan, endpoints, background jobs
streamlit_app.py Optional Streamlit client — talks to the API over HTTP only
tests/ Unit and HTTP-level tests
uv run pytest30 tests, no network and no models required — the HTTP layer is exercised against a stubbed pipeline, and the pure logic (URL parsing, chunk splitting, temporary-directory lifecycle, upload size cap, path-traversal handling) is tested directly.
Deliberate scope choices, not oversights:
- Jobs live in memory. A restart loses them, and with multiple uvicorn
workers a poll can land on a process that doesn't know the job. Production
needs a shared store — Redis or Postgres — and, past a certain load, a real
task queue instead of
BackgroundTasks. - No authentication. The service assumes it sits behind something that handles it.
- Unbounded concurrency. Nothing limits how many jobs run at once; enough simultaneous submissions will exhaust RAM. A worker pool with a bounded queue is the next step.
MIT — see LICENSE.