RAG Chat API

Production-ready Retrieval-Augmented Generation chatbot API — upload documents, ask questions, get cited answers.

Features

• Document ingestion — upload .txt, .md, .csv files with automatic sentence-boundary chunking
• Vector search — OpenAI text-embedding-3-small embeddings stored in ChromaDB
• RAG pipeline — context-aware answers from GPT-4o-mini with source citations
• Chat sessions — conversation history grouped by session ID
• Deduplication — SHA-256 content hashing prevents duplicate uploads
• Async everywhere — fully async FastAPI + SQLAlchemy 2.0 + asyncpg
• Dual database support — SQLite for local dev, PostgreSQL for production
• Docker-ready — one-command deployment with docker compose
• Auto-generated docs — interactive Swagger UI at /docs

Architecture

┌──────────┐     ┌───────────┐     ┌─────────────────┐     ┌──────────────────┐     ┌──────────┐
│          │     │           │     │  Vector Search   │     │ Context Assembly │     │          │
│   User   │────▶│  FastAPI  │────▶│   (ChromaDB)     │────▶│  + System Prompt │────▶│  OpenAI  │
│ Question │     │   Router  │     │                  │     │                  │     │  LLM API │
│          │     │           │     │  top-k chunks    │     │  query + chunks  │     │          │
└──────────┘     └───────────┘     └─────────────────┘     └──────────────────┘     └────┬─────┘
                                                                                         │
┌──────────┐     ┌───────────┐                                                           │
│  Answer  │     │  Persist  │                                                           │
│    +     │◀────│  to chat  │◀──────────────────────────────────────────────────────────┘
│ Sources  │     │  history  │
└──────────┘     └───────────┘

Document Upload Flow:
┌──────┐     ┌──────────┐     ┌───────────┐     ┌──────────────┐     ┌─────────┐
│ File │────▶│ SHA-256   │────▶│ Sentence  │────▶│ Embed chunks │────▶│ Store   │
│ .txt │     │ dedup     │     │ chunking  │     │ (OpenAI)     │     │ ChromaDB│
│ .md  │     │ check     │     │ w/overlap │     │              │     │ + SQL   │
└──────┘     └──────────┘     └───────────┘     └──────────────┘     └─────────┘

Tech Stack

Layer	Technology
Framework	FastAPI 0.115
Language	Python 3.12
LLM	OpenAI GPT-4o-mini
Embeddings	OpenAI text-embedding-3-small
Vector Store	ChromaDB 0.5
ORM	SQLAlchemy 2.0 (async)
Database	PostgreSQL 16 / SQLite
Migrations	Alembic
Validation	Pydantic v2
Containerization	Docker + Docker Compose
Logging	structlog
Linting	Ruff
Testing	pytest + pytest-asyncio

Quick Start

Local Development

git clone https://github.com/your-username/rag-chat-api.git
cd rag-chat-api

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

pip install -e ".[dev]"

cp .env.example .env
# Edit .env and set your OPENAI_API_KEY

uvicorn app.main:app --reload

API is live at http://localhost:8000 — docs at http://localhost:8000/docs

Docker

cp .env.example .env
# Edit .env and set your OPENAI_API_KEY

docker compose up --build

This starts the API on port 8000 and PostgreSQL on port 5432.

API Endpoints

Method	Endpoint	Description
GET	/health	Health check
POST	/documents/upload	Upload a document (file)
GET	/documents/	List all documents
GET	/documents/{doc_id}	Get document by ID
DELETE	/documents/{doc_id}	Delete document and its chunks
POST	/chat/ask	Ask a question (RAG pipeline)
GET	/chat/history/{session_id}	Get chat history for a session

Usage Examples

Upload a document

curl -X POST http://localhost:8000/documents/upload \
  -F "file=@my_notes.txt"

{
  "id": 1,
  "filename": "my_notes.txt",
  "chunk_count": 12,
  "file_size": 4832,
  "created_at": "2026-04-09T12:00:00"
}

Ask a question

curl -X POST http://localhost:8000/chat/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "What are the key points in my notes?", "session_id": "demo-session"}'

{
  "answer": "Based on your documents, the key points are...",
  "sources": [
    {
      "document_id": 1,
      "chunk_index": 3,
      "text_preview": "The main takeaway from the research is...",
      "relevance_score": 0.87
    }
  ],
  "session_id": "demo-session"
}

Get chat history

curl http://localhost:8000/chat/history/demo-session

{
  "session_id": "demo-session",
  "messages": [
    {"role": "user", "content": "What are the key points?", "sources": null, "created_at": "2026-04-09T12:00:00"},
    {"role": "assistant", "content": "Based on your documents...", "sources": [...], "created_at": "2026-04-09T12:00:01"}
  ]
}

List documents

curl http://localhost:8000/documents/

Delete a document

curl -X DELETE http://localhost:8000/documents/1
# 204 No Content

Project Structure

rag-chat-api/
├── app/
│   ├── api/
│   │   ├── chat.py              # Chat endpoints (ask, history)
│   │   ├── documents.py         # Document CRUD endpoints
│   │   └── deps.py              # Dependency injection (DI container)
│   ├── models/
│   │   ├── document.py          # Document SQLAlchemy model
│   │   └── message.py           # ChatMessage SQLAlchemy model
│   ├── schemas/
│   │   ├── chat.py              # Chat request/response schemas
│   │   ├── common.py            # Shared schemas (health, errors)
│   │   └── document.py          # Document request/response schemas
│   ├── services/
│   │   ├── document_service.py  # Document processing + chunking
│   │   ├── llm_client.py        # OpenAI async wrapper
│   │   ├── rag_service.py       # RAG orchestration
│   │   └── vector_store.py      # ChromaDB wrapper
│   ├── config.py                # Settings via pydantic-settings
│   ├── database.py              # Async engine + session factory
│   ├── exceptions.py            # Custom exception hierarchy
│   └── main.py                  # FastAPI app + lifespan
├── alembic/                     # Database migrations
├── tests/                       # Test suite
├── .env.example                 # Environment template
├── Dockerfile                   # Container image
├── docker-compose.yml           # Full stack (app + PostgreSQL)
└── pyproject.toml               # Dependencies + tool config

How RAG Works

Retrieval-Augmented Generation combines document retrieval with LLM generation to produce grounded, source-cited answers instead of relying solely on the model's training data.

1. INGEST    User uploads a document
             → Text is split into overlapping chunks (500 chars, 50 char overlap)
             → Each chunk is embedded via OpenAI text-embedding-3-small
             → Embeddings are stored in ChromaDB alongside metadata

2. RETRIEVE  User asks a question
             → The question is embedded using the same model
             → ChromaDB performs cosine similarity search
             → Top-k most relevant chunks are returned

3. GENERATE  Retrieved chunks are assembled into a context block
             → Context + question are sent to GPT-4o-mini
             → The system prompt instructs the model to cite sources
             → The answer + source references are returned to the user

Why RAG over fine-tuning?

• Works with private/dynamic data without retraining
• Sources are traceable — every answer points back to specific document chunks
• Cost-effective — embedding is cheap; only the query hits the LLM
• Data stays current — upload new documents anytime, no retraining pipeline

Configuration

Variable	Default	Description
DATABASE_URL	sqlite+aiosqlite:///./rag_chat.db	Database connection string
OPENAI_API_KEY	—	OpenAI API key (required)
OPENAI_MODEL	gpt-4o-mini	Chat completion model
OPENAI_EMBEDDING_MODEL	text-embedding-3-small	Embedding model
CHROMA_PERSIST_DIR	./chroma_data	ChromaDB persistence directory
LOG_LEVEL	INFO	Logging level
CHUNK_SIZE	500	Max characters per text chunk
CHUNK_OVERLAP	50	Overlap between consecutive chunks
MAX_CONTEXT_CHUNKS	5	Chunks sent to LLM as context

Testing

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run with coverage
coverage run -m pytest
coverage report -m

# Lint
ruff check .
ruff format --check .

License

MIT