MedRag: Medical Document Retrieval-Augmented Generation

MedRag is a powerful Retrieval-Augmented Generation (RAG) system specifically designed for querying and analyzing medical documents. It allows users to upload custom PDF, DOCX, or TXT files and ask questions, receiving answers grounded only in the provided document's content.

This project is built using the LangChain framework and leverages modern embedding models and vector databases to ensure high-accuracy, context-specific responses from large medical texts.

---

Features

• Context-Specific QA: Acts as a medical assistant, answering questions strictly based on the provided document context to prevent hallucinations or use of external knowledge.
• Multi-Format Document Support: Easily ingest PDF, TXT, and DOCX files (legacy .doc is not supported).
• Flexible Vector Store Options: Supports integration with multiple vector databases:
  • FAISS (default for local, fast indexing)
  • ChromaDB
  • Pinecone (requires API Key)
• LLM Fallback Chain: Answers are generated by Google Gemini; if Gemini fails (quota, rate limit, outage, timeout), the same question is sent to Groq. If both fail, the most relevant document excerpts are returned instead.
• Retrieval Customization: Configurable document chunking (chunk_size, chunk_overlap) and retrieval size (top_k_retrieval).
• LangChain QA Chain Option: Includes an alternative mode to switch to a standard RetrievalQA chain implementation.

---

Web App (FastAPI + React + PostgreSQL)

The same RAG pipeline is also available as a web application:

Part	Stack	Location
Backend	FastAPI, SQLAlchemy, LangChain (splitting + chat models), sentence-transformers	backend/
Database / vector store	PostgreSQL 16 with pgvector (HNSW index, cosine distance)	docker-compose.yml
Frontend	React + TypeScript (Vite), served by nginx in Docker	frontend/

Documents, chunks with their embeddings, and question history are all stored in Postgres, so they persist across restarts and you can search several documents at once.

What the web app does:

• Accounts: everyone signs in, and each user's documents, chats and history are private to them.
• Chats: questions are grouped into chats, each titled after its first question; rename, delete and page through them (30 at a time).
• Upload PDF, DOCX and TXT files. Scanned PDFs (pages with no text layer) are read with Tesseract OCR, and the document list marks which ones were OCR'd.
• Streamed answers: the retrieved sources appear first, then the answer as it is generated, labelled with the model that wrote it.
• Gemini → Groq fallback: if a provider fails before it starts answering (quota, outage, timeout), the next one takes over. If all fail, the most relevant excerpts are shown instead.
• Similarity cutoff: when nothing in the documents is close to the question (MIN_SIMILARITY), it is refused straight away without spending an LLM call.
• Search scope: tick documents to search only those.
• History: each chat loads 20 questions at a time ("Load earlier questions"); single questions can be deleted.

Accounts

• The first account to register takes over everything created before accounts existed: existing documents, and all earlier questions in a chat called "Earlier questions". After upgrading, register your own account first.

• Passwords are hashed with Argon2. Sign-in creates a server-side session, kept in an httpOnly SameSite=Lax cookie that page scripts can't read. Sign-out ends it. Sessions last SESSION_TTL_HOURS, and only a hash of each token is stored.

• API clients (scripts, the evaluation harness) send the token from POST /api/auth/login as Authorization: Bearer <token>. The Swagger page at /docs has an Authorize button for it.

• After LOGIN_MAX_FAILURES wrong passwords for an email from one address, sign-in is refused for LOGIN_WINDOW_MINUTES. The counter is per backend process.

• With ALLOW_REGISTRATION=false, accounts are created from the command line:

  docker compose exec backend python -m app.cli create-user someone@example.org --name "Dr Someone"
  docker compose exec backend python -m app.cli reset-password someone@example.org   # also signs them out everywhere
  docker compose exec backend python -m app.cli list-users

• DELETE /api/auth/me (with the password) deletes an account and all its documents, chats and sessions.

• Behind HTTPS, set COOKIE_SECURE=true so the session cookie is never sent over plain HTTP.

Run with Docker

cp .env.example .env        # then set GEMINI_API_KEY and GROQ_API_KEY
docker compose up --build

• App: http://localhost:8080
• API docs (Swagger): http://localhost:8000/docs

The backend image includes Tesseract (English) for OCR. For other languages, add the matching tesseract-ocr-<lang> package in backend/Dockerfile and set OCR_LANGUAGE (e.g. eng+fra).

The first start downloads the all-MiniLM-L6-v2 embedding model from Hugging Face, and it is cached in a Docker volume after that. With no LLM key set, answers are retrieval-only excerpts.

Run locally for development

# Database: Postgres 16 with pgvector on localhost:5433
docker compose up db -d

# Backend
cd backend
pip install -r requirements-dev.txt
export DATABASE_URL=postgresql+psycopg://postgres:postgres@localhost:5433/medrag
uvicorn app.main:app --reload            # http://localhost:8000
# OCR outside Docker needs the tesseract binary on PATH (apt install tesseract-ocr / brew install tesseract /
# choco install tesseract); without it, PDFs with a text layer still work and scanned ones fail with a clear error

# Frontend (proxies /api to the backend)
cd frontend
npm install
npm run dev                              # http://localhost:5173

API

Everything except /api/health and /api/auth/{config,register,login} needs a signed-in user, and only ever sees that user's data. Requests for another user's document, chat or question get 404.

Method	Path	Purpose
POST	/api/auth/register	{"email", "password", "name"?}: create an account (password at least 8 characters) and sign in. 409 if the email is taken, 403 if registration is off.
POST	/api/auth/login	{"email", "password"}: sets the session cookie and returns {user, token, expires_at}. 401 on a wrong password, 429 after too many.
POST	/api/auth/logout	End the current session.
GET / DELETE	/api/auth/me	The signed-in user, or delete the account and all its data (body {"password"}).
GET	/api/auth/config	Public: whether registration is open, and the minimum password length.
GET / POST	/api/chats?limit=30&offset=0	Your chats, most recently active first (total in X-Total-Count), or create one ({"title"?}).
GET / PATCH / DELETE	/api/chats/{id}	Get, rename ({"title"}) or delete a chat with its questions.
GET	/api/chats/{id}/messages?limit=20&offset=0	A chat's questions and answers, newest first (total in X-Total-Count).
POST	/api/documents	Upload a PDF, DOCX or TXT file (multipart field file). Processing runs in the background and the status goes processing → ready / failed.
GET	/api/documents?limit=100&offset=0	Documents, newest first, with status, chunk count and ocr_pages. The total is in the X-Total-Count header.
GET / DELETE	/api/documents/{id}	Get a document, or delete it along with its chunks.
POST	/api/query	{"question": "...", "chat_id": 3, "document_ids": [1, 2], "top_k": 4}. Without chat_id, a new chat is started (titled after the question). Returns the answer, the LLM that actually answered (e.g. groq:... after a Gemini failure; similarity_cutoff or retrieval_only when no LLM was used), and the sources. document_ids and top_k are optional.
POST	/api/query/stream	Same body, answered as server-sent events: sources (with the chat_id), then token events as the answer is generated, then done with the saved result (same shape as /api/query).
POST	/api/search	Same body; returns the passages that would be retrieved, without calling an LLM or saving history.
GET	/api/queries?limit=20&offset=0	All your questions across chats, newest first. The total is in the X-Total-Count header.
DELETE	/api/queries/{id}, /api/queries	Delete one question, or all your chats and questions (documents are kept).
GET	/api/info, /api/health	Active configuration and counts, and a liveness check.

Configuration

The backend reads environment variables (or backend/.env):

Variable	Default	Description
DATABASE_URL	postgresql+psycopg://postgres:postgres@localhost:5433/medrag	Postgres connection string.
GEMINI_API_KEY / GROQ_API_KEY	(empty)	Provider keys. A provider without a key is skipped.
LLM_PROVIDERS	gemini,groq	Fallback order. If one provider errors or returns nothing, the next is tried. none disables generation.
GEMINI_MODEL / GROQ_MODEL	gemini-3.8-flash / openai/gpt-oss-120b	Chat model for each provider.
LLM_TIMEOUT / LLM_MAX_RETRIES	30 / 1	Per-provider timeout (seconds) and retries before falling back.
EMBEDDING_MODEL / EMBEDDING_DIM	all-MiniLM-L6-v2 / 384	Must match each other. Changing them requires re-uploading documents.
CHUNK_SIZE / CHUNK_OVERLAP / TOP_K	512 / 128 / 4	Chunking and retrieval.
MIN_SIMILARITY	0.35	Refuse without calling the LLM when the best passage's cosine similarity is below this. 0 disables it. Tuned on the evaluation sets; see eval/README.md. Tied to the embedding model, so re-tune it if you change EMBEDDING_MODEL.
OCR_ENABLED / OCR_LANGUAGE / OCR_DPI	true / eng / 300	OCR for PDF pages without a text layer.
MAX_UPLOAD_MB	25	Upload size limit.
ALLOW_REGISTRATION	true	Let anyone create an account from the sign-in page. With false, use python -m app.cli create-user.
SESSION_TTL_HOURS	168	How long a sign-in lasts.
COOKIE_SECURE	false	Set true when served over HTTPS.
LOGIN_MAX_FAILURES / LOGIN_WINDOW_MINUTES	5 / 15	Sign-in throttling per email and client address.

Tests

cd backend
pytest        # needs Postgres with pgvector; set DATABASE_URL to a disposable database (default: medrag_test)

cd frontend
npm test      # Vitest + Testing Library in jsdom; also runs during the frontend Docker build

The backend tests use a stand-in embedder and LLM, so no model download or API key is needed. They drop and recreate all tables in the target database. The OCR test is skipped when Tesseract is not installed; it runs inside the backend image:

docker compose exec db createdb -U postgres medrag_test   # once
docker compose run --rm -v ${PWD}/backend/tests:/app/tests -e DATABASE_URL=postgresql+psycopg://postgres:postgres@db:5432/medrag_test backend sh -c "pip install pytest httpx && pytest -q"

Evaluation

eval/ measures a running instance end to end: a quick hand-written smoke set, the expert-labelled PubMedQA set (1,000 questions over real PubMed abstracts), and a CSV workflow for clinicians to grade answers.

Results

Baseline on 2026-10-04 (all-MiniLM-L6-v2 embeddings, chunk size 512, top_k 4, MIN_SIMILARITY 0.35, Gemini gemini-3.8-flash → Groq openai/gpt-oss-120b). Full reports: smoke, PubMedQA.

	Smoke set	PubMedQA
Documents / questions	5 / 33	900 abstracts (+100 held out) / 1,000
Right source ranked first	100% (25/25)	98.0% (882/900)
Right source in top 4 (what the LLM sees)	100% (25/25)	99.4% (895/900)
Answers correct	100% (25/25, every key fact present)	76.9% (40/52) agree with the expert yes/no/maybe label
Answerable questions refused	0% (0/25)	3.7% (2/54)
Unanswerable questions refused	100% (8/8)	83.3% (5/6) held-out questions
Refused by the cutoff, no LLM call	6 of 33	1 of 100 held-out; 2 of 900 answerable
Latency p50 / p95	1.05 s / 1.30 s	1.53 s / 1.78 s

• PubMedQA answers come from a sample of 60 questions, so treat 76.9% as roughly ±11 points. For context, the PubMedQA paper reports 78% for a single human annotator.
• Free-tier Gemini quotas meant every LLM answer in these runs came from the Groq fallback.
• Most PubMedQA disagreements (9 of 12) are the model answering "maybe" where the expert said yes or no.
• The smoke set was written for this project and is deliberately easy; it catches regressions rather than measuring quality. See eval/README.md for the failure analysis and how MIN_SIMILARITY was chosen.

---

Notebook: Installation and Setup

1. Dependencies

This project requires Python and uses several key libraries. The dependencies can be installed using pip.

pip install langchain langchain-classic langchain-community langchain-google-genai langchain-groq
pip install faiss-cpu sentence-transformers chromadb pypdf docx2txt torch
# Optional dependencies if using other models/stores
# pip install pinecone

The necessary core libraries found in the project include:

• langchain
• langchain-google-genai
• langchain-groq
• langchain-community
• faiss-cpu
• sentence-transformers
• chromadb

2. API Keys (If needed)

The system is designed to use external LLMs and vector stores, which require API keys. Set up your keys as environment variables or pass them directly as a dictionary to the MedicalDocumentRAG class initialization.

Service	Key Variable Name
Google Gemini	GEMINI_API_KEY (or GOOGLE_API_KEY)
Groq	GROQ_API_KEY
Pinecone	PINECONE_API_KEY

The notebook reads keys from environment variables and only prompts for the ones that are not set.

---

Usage

The project is structured as a Jupyter Notebook (MedRag (1).ipynb) and can be run step-by-step.

Example Configuration from the notebook

config = RAGConfig( chunk_size=512, chunk_overlap=128, top_k_retrieval=4, temperature=0.1, vector_store_type="faiss" # Options: faiss, chromadb, pinecone ) rag_system = MedicalDocumentRAG(config=config, api_keys=api_keys)

Step 1: Configuration

Initialize the system by defining a configuration, including the vector store type (default is faiss) and retrieval settings.

Step 2: Document Loading

You can choose to upload your own document or use a sample text for testing:

1. Upload: Enter the path to your document (.pdf, .docx, or .txt) when prompted.
2. Sample Text: The notebook includes a built-in sample text on Chronic Fatigue Syndrome (CFS) for quick testing.

The system automatically performs document splitting and chunking using the configured parameters.

Step 3: Create Embeddings and Index

After loading the document, create the vector index. The system uses a HuggingFace Embedding Model (all-MiniLM-L6-v2 by default) for creating embeddings.

Runs automatically in the main loop if 'p' (Process) is selected

rag_system.create_embeddings_and_index()

Step 4: Ask Questions (QA Loop)

Enter the main loop to ask medical questions about the document.

The output will provide the Answer, the LLM Type used, the Number of Retrieved Chunks, and the Source Documents used to formulate the answer, including page and source file information.

> Enter your question: What are the three core diagnostic symptoms for Chronic Fatigue Syndrome?

Type stats to see system statistics, chain to toggle the LangChain QA chain, or q to quit.

Note: If no LLM is initialized, the system defaults to a retrieval-only mode and will return the most relevant document excerpts without generating a synthetic answer.

---

RAG Configuration Details

The RAGConfig class allows you to fine-tune the system's performance:

Parameter	Default Value	Description
embedding_model	all-MiniLM-L6-v2	The HuggingFace model for creating embeddings.
vector_store_type	"faiss"	Vector store backend: faiss, chromadb, or pinecone.
chunk_size	1000	The maximum size of text chunks for indexing (the demo uses 512).
chunk_overlap	200	The overlap between consecutive text chunks (the demo uses 128).
top_k_retrieval	5	The number of most relevant documents to retrieve for the LLM (the demo uses 4).
temperature	0.1	The LLM generation temperature.
max_tokens	500	Max response length for LLM generation.

llm_providers (default ("gemini", "groq")) sets the fallback order; gemini_model / groq_model choose the models. Every provider with a key is tried in turn, and the answer shows which one replied. The Pinecone index is named medical-rag-index.

---

Contributing

Contributions are welcome! If you have suggestions for new LLMs, vector stores, or document loaders, please submit a pull request.