RAGOps — Production RAG Pipeline

RAGOps is a production-oriented Retrieval-Augmented Generation (RAG) platform. It ingests documents, indexes them with hybrid vector + keyword search, and answers questions using retrieved context and a hosted LLM — all orchestrated as containerized microservices with persistent storage, caching, and observability.

Overview

RAGOps solves a common production problem: how to let an LLM answer questions grounded in your own documents, not just its training data.

The system provides:

Document ingestion — JSON text, single PDF, and batch PDF pipelines
Hybrid retrieval — BM25 keyword search combined with semantic vector search (Meilisearch)
RAG Q&A — retrieve relevant chunks, synthesize an answer via Groq (through LiteLLM)
Object storage — raw files persisted in MinIO (S3-compatible)
Metadata catalog — document records tracked in PostgreSQL
Performance layer — Redis caching for embeddings and RAG responses
Observability — Prometheus metrics, Grafana dashboards, optional LangSmith tracing

The API is the primary interface. Interactive documentation is available at /docs once the stack is running.

Architecture

┌──────────────┐     ┌─────────────────────────────────────────────────────────────┐
│   Client     │────▶│                    FastAPI Backend (:18000)                 │
│  (curl/SDK)  │     │  ingest · search · RAG · upload · metadata · metrics        │
└──────────────┘     └──────┬──────────┬──────────┬──────────┬──────────┬──────────┘
                            │          │          │          │          │
              ┌─────────────┘          │          │          │          └─────────────┐
              ▼                        ▼          ▼          ▼                        ▼
     ┌────────────────┐      ┌──────────────┐ ┌────────┐ ┌────────┐      ┌────────────────┐
     │  Meilisearch   │      │   LiteLLM    │ │ Redis  │ │ MinIO  │      │   PostgreSQL   │
     │  documents +   │      │   Proxy      │ │ Cache  │ │  S3    │      │   metadata     │
     │  chunks index  │      └──────┬───────┘ └────────┘ └────────┘      └────────────────┘
     └────────────────┘             │
                                    ▼
                          ┌─────────────────┐
                          │  Groq LLM       │       ┌────────────────────┐
                          │  (external)     │───────│  TEI Embeddings    │
                          └─────────────────┘       │ (CPU, 384-dim)     │
                                                    └────────────────────┘

     Observability:  Prometheus (:9090)  →  Grafana (:3000)

Data stores and responsibilities

Store	Role	Production purpose
Meilisearch	Search index (`documents` + `chunks` with vectors)	Fast hybrid retrieval at query time
MinIO	Raw file storage (`s3://documents/...`)	Durable object store for uploaded/ingested files
PostgreSQL	Document metadata catalog	Auditable record of what was uploaded and where
Redis	Embedding + RAG response cache	Reduced latency and lower LLM/embedding cost

Request flows

Ingestion

Document/PDF → Chunking → Embeddings (TEI via LiteLLM) → Meilisearch
              └→ MinIO (raw file) + Postgres (metadata)   [PDF/upload paths]

RAG query

Question → Query embedding → Hybrid search (Meilisearch) → Context assembly → LLM (Groq) → Answer + sources

Production features

Area	Implementation
Service isolation	Each concern (search, embeddings, LLM, storage) runs as its own container
Health checks	All core services expose health endpoints; backend validates embedding pipeline on `/health`
Restart policies	`unless-stopped` / `always` on stateful and critical services
Persistent volumes	Meilisearch, Redis, TEI model cache, Postgres, and MinIO data survive restarts
Model abstraction	LiteLLM proxy decouples the backend from specific LLM/embedding providers
Caching	Redis caches embeddings (1 h TTL) and RAG answers (10 min TTL)
Metrics	Prometheus scrapes `/metrics` from the FastAPI instrumentator
Tracing	LangSmith callbacks configured in LiteLLM for LLM observability
CPU deployment	TEI serves `all-MiniLM-L6-v2` embeddings without a GPU
Index bootstrap	`meili-init` job configures Meilisearch indexes and vector settings on startup

Prerequisites

Requirement	Notes
Docker & Docker Compose	Latest stable versions
4 GB+ RAM	8 GB recommended (TEI model download on first boot)
2 GB+ disk	Models, indexes, and object storage
Groq API key	Required for LLM completions (console.groq.com)
LangSmith API key	Optional; enables tracing and RAGAS evaluation

Quick start

1. Configure environment

cp .env.example .env
# Edit .env — at minimum set GROQ_API_KEY, MEILI_KEY, and PROXY_KEY

2. Build images

The backend image is built locally from backend/Dockerfile. Other services use public images from Docker Hub / GHCR.

make build
# or: docker compose build

3. Start the stack

make up
# or: docker compose up -d

First startup may take several minutes while TEI downloads the embedding model.

4. Verify health

curl -s http://localhost:18000/health | jq .
docker compose ps

Expected health response:

{
  "status": "healthy",
  "embeddings_available": true,
  "embedding_dimensions": 384
}

5. Load sample documents (optional)

docker compose --profile manual run --rm seed

6. Ask a question

curl -s -X POST http://localhost:18000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "What is retrieval-augmented generation?", "k": 5}' | jq .

Configuration

All configuration is driven by environment variables. See .env.example for the full template.

Required variables

Variable	Description
`GROQ_API_KEY`	Groq API key for LLM completions
`MEILI_KEY`	Meilisearch master key
`PROXY_KEY`	LiteLLM proxy master key
`LITELLM_MODEL`	Chat model alias (default: `groq-llama3`)
`EMBEDDING_MODEL_NAME`	Embedding model alias (default: `local-embeddings`)

Infrastructure variables (Docker defaults)

Variable	Default	Service
`MEILI_URL`	`http://meilisearch:7700`	Meilisearch
`PROXY_URL`	`http://litellm:4000`	LiteLLM
`REDIS_URL`	`redis://redis:6379`	Redis
`POSTGRES_URL`	`postgresql://ragops:ragops@postgres:5432/metadata`	PostgreSQL
`MINIO_ENDPOINT`	`http://minio:9000`	MinIO
`MINIO_BUCKET`	`documents`	MinIO bucket name
`EMBED_DIM`	`384`	Must match TEI model output

Model routing is defined in litellm/config.yaml. Model aliases in .env must match the model_name entries in that file.

Security: Never commit .env to version control. Rotate MEILI_KEY, PROXY_KEY, and MinIO credentials before any non-local deployment.

Services

Service	Host port	Health	Description
backend	18000	`GET /health`	FastAPI application — main API entry point
meilisearch	7700	`GET /health`	Hybrid search engine (documents + chunks indexes)
litellm	4000	`GET /health`	LLM and embedding proxy (Groq + TEI)
tei-embeddings	— (internal)	`GET /health`	CPU embedding inference (`all-MiniLM-L6-v2`)
redis	6379	TCP	Embedding and RAG response cache
postgres	5432	`pg_isready`	Document metadata catalog
minio	9000 / 9001	`GET /minio/health/live`	Object storage / web console
prometheus	9090	—	Metrics collection
grafana	3000	—	Dashboards (`admin` / `admin`)
meili-init	—	—	One-shot index configuration job

Useful URLs after startup:

URL	Purpose
http://localhost:18000/docs	Swagger UI — interactive API
http://localhost:18000/metrics	Prometheus metrics endpoint
http://localhost:3000	Grafana dashboards
http://localhost:9001	MinIO console

API reference

Health and administration

Method	Endpoint	Description
`GET`	`/health`	Service health + embedding pipeline check
`GET`	`/stats`	Meilisearch document and chunk index statistics
`POST`	`/init-index`	Create/reconfigure Meilisearch indexes with vector settings
`POST`	`/test-embeddings`	Validate embedding generation

Ingestion (search index)

Method	Endpoint	Description
`POST`	`/ingest`	Ingest JSON documents (`[{id, text, metadata}]`)
`POST`	`/ingest-pdf`	Ingest a single PDF (chunk, embed, index + store)
`POST`	`/ingest-pdf-batch`	Batch PDF ingestion

File storage and metadata

Method	Endpoint	Description
`POST`	`/upload`	Upload a file to MinIO and record metadata in Postgres
`GET`	`/metadata/{document_id}`	Retrieve document metadata by ID

Search and generation

Method	Endpoint	Description
`POST`	`/search`	Full RAG — retrieve chunks and generate an LLM answer
`POST`	`/search-chunks`	Retrieve relevant chunks without LLM synthesis
`POST`	`/search-direct`	Direct document-level search
`POST`	`/search-rerank`	Hybrid search with cosine-similarity reranking
`POST`	`/chat`	Direct LLM chat (no retrieval)

Example: ingest text

curl -X POST http://localhost:18000/ingest \
  -H "Content-Type: application/json" \
  -d '[{
    "id": "doc-001",
    "text": "Retrieval-Augmented Generation combines search with language models.",
    "metadata": {"title": "RAG Overview", "category": "ai"}
  }]'

Example: ingest a PDF

curl -X POST http://localhost:18000/ingest-pdf \
  -F "file=@pdf_files/linear_algebra.pdf"

Example: RAG query

curl -X POST http://localhost:18000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "Explain matrix multiplication", "k": 5, "use_embeddings": true}'

Response shape:

{
  "answer": "...",
  "chunks": [{"id": "...", "document_id": "...", "content": "...", "chunk_index": 0}],
  "total_chunks_found": 5,
  "cached": false,
  "search_method": "hybrid"
}

Operations

Makefile commands

make help        # Show available commands
make build       # Build Docker images
make up          # Start all services
make down        # Stop all services
make restart     # Restart the stack
make logs        # Follow backend logs
make test        # Run comprehensive test suite
make clean       # Remove containers and volumes
make dev-reset   # Full reset: wipe volumes, restart, re-init indexes

Docker Compose profiles

Profile	Command	Purpose
`manual`	`docker compose --profile manual run --rm seed`	Load sample documents
`eval`	`docker compose --profile eval run --rm ragas_eval`	Run RAGAS evaluation

Database schema

PostgreSQL is initialized from db/init.sql on first container start:

CREATE TABLE documents (
  document_id   SERIAL PRIMARY KEY,
  filename      TEXT NOT NULL,
  upload_time   TIMESTAMP DEFAULT NOW(),
  chunk_count   INTEGER,
  embedding_model TEXT,
  minio_path    TEXT NOT NULL
);

If the Postgres volume was created before db/init.sql existed, recreate it with make dev-reset or docker compose down -v.

Monitoring and observability

Prometheus

Scrapes the backend /metrics endpoint every 15 seconds. Configuration: monitoring/prometheus/prometheus.yml.

Grafana

Pre-provisioned datasource and dashboard. Access at http://localhost:3000 (default credentials: admin / admin). Dashboard: monitoring/grafana/dashboards/simple_rag_dashboard.json.

LangSmith

LiteLLM is configured with LangSmith success/failure callbacks. Set LANGSMITH_API_KEY, LANGSMITH_PROJECT, and LANGSMITH_BASE_URL in .env to enable distributed tracing of LLM calls.

Structured logging

The backend emits structured JSON logs via the ragops logger. View with:

docker compose logs -f backend

Testing

# Comprehensive integration tests (requires running stack)
make test

# Metadata / MinIO / Postgres tests
pytest tests/test_metadata.py

# Chunking validation
python tests/chunking_validation.py

# Quick RAG smoke test
python tests/test_rag_quick.py

Production deployment guide

Before going live

Rotate all secrets — MEILI_KEY, PROXY_KEY, MinIO credentials, Postgres password
Restrict port exposure — bind internal services (Redis, Postgres, Meilisearch) to private networks only
Use external secrets management — Docker secrets, Vault, or cloud provider secret stores instead of plain .env files
Enable TLS — terminate HTTPS at an ingress (Traefik, Nginx, cloud load balancer)
Back up persistent volumes — Meilisearch indexes, Postgres, and MinIO data
Set resource limits — define CPU/memory limits per service in Compose or your orchestrator

Scaling considerations

Component	Strategy
backend	Horizontally scalable behind a load balancer (stateless)
meilisearch	Single instance or Meilisearch Cloud for HA
redis	Single instance; use Redis Sentinel/Cluster for HA
tei-embeddings	Scale replicas; consider GPU instances for higher throughput
litellm	Scale replicas; configure provider rate limits and fallbacks
postgres	Managed database service recommended for production
minio	Distributed MinIO or cloud S3 for durability

Recommended resource allocation

Service	Memory	CPU
backend	1–2 GB	1 core
meilisearch	1–2 GB	1 core
tei-embeddings	2–4 GB	2 cores
litellm	512 MB–1 GB	0.5 core
postgres	512 MB–1 GB	0.5 core
minio	512 MB	0.5 core
redis	256 MB	0.25 core

Backup procedures

# Meilisearch documents dump
curl -H "Authorization: Bearer $MEILI_KEY" \
  "http://localhost:7700/indexes/documents/documents" > backup_documents.json

# Meilisearch chunks dump
curl -H "Authorization: Bearer $MEILI_KEY" \
  "http://localhost:7700/indexes/chunks/documents" > backup_chunks.json

# Postgres metadata
docker compose exec postgres pg_dump -U ragops metadata > backup_metadata.sql

Project structure

.
├── backend/
│   ├── app/
│   │   ├── api/              # FastAPI route handlers
│   │   ├── core/             # Config, clients, logging
│   │   ├── eval/             # RAGAS evaluation
│   │   ├── models/           # Pydantic request/response schemas
│   │   ├── services/         # Business logic (ingestion, RAG, storage)
│   │   └── utils/            # Caching, hashing
│   ├── Dockerfile
│   ├── requirements.txt
│   └── seed_data.py          # Sample document loader
├── db/
│   └── init.sql              # Postgres schema (runs on first start)
├── litellm/
│   └── config.yaml           # LLM routing, caching, LangSmith callbacks
├── monitoring/
│   ├── prometheus/
│   └── grafana/
├── pdf_files/                # Sample PDFs for testing
├── scripts/
│   └── meili-init.sh         # Meilisearch index bootstrap
├── tests/
├── docker-compose.yml
├── Makefile
├── .env.example
└── README.md

Troubleshooting

Symptom	Likely cause	Fix
`/health` returns `embeddings_available: false`	TEI still starting or LiteLLM misconfigured	Wait 2–3 min; check `docker compose logs tei-embeddings litellm`
`relation "documents" does not exist`	Postgres volume created before `db/init.sql`	`make dev-reset` or `docker compose down -v && docker compose up -d`
`500` on `/upload` or `/ingest-pdf`	MinIO or Postgres unreachable	Verify `docker compose ps`; check backend logs
Empty search results	No documents ingested	Run seed profile or ingest documents manually
LLM errors (`502`)	Invalid or missing `GROQ_API_KEY`	Verify key in `.env` and restart LiteLLM
Slow first query	Cold start: model load + cache miss	Expected; subsequent queries benefit from Redis cache

# Diagnose a failing service
docker compose ps
docker compose logs backend --tail 50
docker compose logs tei-embeddings litellm postgres minio --tail 30

Technology stack

Layer	Technology
API	FastAPI 0.112, Uvicorn, Pydantic v2
Search	Meilisearch 1.16 (hybrid BM25 + vector)
Embeddings	TEI (`sentence-transformers/all-MiniLM-L6-v2`, 384-dim)
LLM	Groq `llama-3.1-8b-instant` via LiteLLM
Object storage	MinIO (S3-compatible)
Metadata DB	PostgreSQL 15
Cache	Redis 7
PDF processing	LangChain PyPDFLoader + RecursiveCharacterTextSplitter
Metrics	Prometheus + Grafana
Evaluation	RAGAS
Runtime	Python 3.11, Docker Compose

License

This project is licensed under the MIT License. See LICENSE for the full text.

Latest commit

History

Folders and files

Repository files navigation