ResearchPaperRAG is an open-source Retrieval-Augmented Generation (RAG) platform designed for researchers, computer scientists, academics, and students. It automates multi-source academic paper discovery, enforces a strict 3-year publication window (2024–2026) to capture cutting-edge literature, indexes text layers into local vector stores using zero-cost CPU embeddings, and synthesizes structured 8-Layer Research Analysis & Gap Reports.
The platform incorporates an interactive Research Gap Analyzer SPA powered by a high-performance FastAPI / Gunicorn backend with multi-provider failover across 27+ AI models.
- GitHub Repository: https://github.com/AishikTokdar/ResearchPaperRAG
- Key Capabilities & Features
- Structured 8-Layer Literature Synthesis Framework
- System Architecture & Data Flow
- Component Breakdown
- Obtaining & Configuring Free AI API Keys
- Complete Environment Configuration Reference (.env)
- Local Quickstart & Execution Guide
- Production Deployment with Gunicorn & Uvicorn
- Docker & Containerization Guide
- Cloud Deployment Options (Hugging Face Spaces Backend & Vercel / Cloudflare Pages Frontend)
- REST API Reference & OpenAPI Specification
- Troubleshooting & FAQs
- Tech Stack Summary
- License & Attribution
ResearchPaperRAG conducts concurrent, real-time searches across 6 open-access literature databases without requiring paid subscriptions:
- arXiv API: Computer Science, Artificial Intelligence, Machine Learning, Physics, Mathematics, and Quantitative Biology.
- Crossref API: Global DOI metadata registry covering peer-reviewed conference proceedings and journal articles.
- Semantic Scholar API: Academic graph database providing paper abstracts, citation metrics, and author affiliations.
- OpenAlex API: Fully open global bibliometric database indexing over 250 million scholarly entities.
- PubMed API: National Library of Medicine repository for biomedical, healthcare, and life sciences literature.
- DOAJ API: Directory of Open Access Journals across all scientific disciplines.
Research results are automatically validated against publication dates and filtered strictly to the current date's 3-year publication window (2024, 2025, 2026). This filters out outdated methodologies and forces the gap analysis engine to focus exclusively on contemporary state-of-the-art research.
Users can select up to 5 research papers (or attach custom PDF files up to 50 MB cumulative size) per gap analysis run. A custom _get_balanced_documents retriever guarantees that chunks from every selected paper are represented in the RAG context, eliminating single-paper bias during multi-paper comparison questions.
Every fetched paper in the interactive search list includes direct external links (url / pdf_url / doi). Clicking any paper card immediately opens the original publisher document or PDF in a new browser tab.
Following the 8-layer report synthesis, users can conduct follow-up Q&A strictly grounded in the ingested paper texts. Inline citations clean out raw Markdown formatting symbols and render as styled inline pills. Re-running gap analysis on new papers automatically resets the follow-up chat session, and a manual "Clear Chat" button allows instant thread clearing.
- Print PDF Export (.pdf): Generates a clean print-styled document with custom CSS table styling and triggers native browser printing.
- Raw Markdown Export (.md): Saves full report markdown with H2 section card dividers.
- Plain Text Export (.txt): Strips markdown symbols for unformatted text logging.
Redesigned model management bar eliminates raw text input boxes and manual API key fields. Users select from a pre-configured 27-model registry, click "Save Model", and receive instant toast notifications. Switching models automatically resets both the 8-layer gap report and the follow-up chat session.
Supplies comma-separated or space-separated API keys (KEY_1,KEY_2,KEY_3) across all AI providers (Google Gemini, Groq, Cerebras, SambaNova, Hugging Face, OpenRouter). When a key encounters 429 Rate Limits, the engine rotates instantly to the next key in the pool.
If a primary model fails, the fallback engine automatically traverses all 30+ AI models across all credentialed providers in priority order. If all models and keys fail, the UI displays a clean, user-friendly Markdown notice without leaking raw 500 error tracebacks.
- In-Memory Model Health Tracker: Automatically tracks consecutive failure counts per model ID with an automatic threshold (
MAX_CONSECUTIVE_FAILURES = 3). Resets failure counts on successful invocations and flags models/providers upon repeated HTTP/rate-limit exceptions. - Runtime Summary Endpoint (
GET /runtime-summary): Returns overall system status (ok,degraded,error) and provider statuses (working,partial,unavailable). - Interactive Status Dashboard (
/api-status): Dark-mode SPA page featuring real-time provider health badges, default model indicator, vector chunking metrics (1000chars chunk /200chars overlap), resolved base URL, and direct OpenAPI docs link.
When papers are ingested, ResearchPaperRAG executes cross-document retrieval and passes chunks through an 8-layer prompt orchestration engine:
| Layer | Section Name | Analytical Focus & Content Output |
|---|---|---|
| 1 | Literature Summary | Core research objectives, primary datasets, model architectures, and key empirical takeaways from each selected paper. |
| 2 | Trend Detection | Architectural shifts, dataset evolutions, and algorithmic trends observed across the 2024–2026 publication timeline. |
| 3 | Common Methods | Shared baseline models, evaluation metrics (F1, BLEU, ROUGE, Accuracy), data preprocessing pipelines, and loss functions. |
| 4 | Limitations | Methodological bottlenecks, hardware/compute constraints, domain generalization failures, and dataset distribution bias. |
| 5 | Contradictions | Conflicting findings, empirical discrepancies, and opposing experimental results between the analyzed papers. |
| 6 | Research Gaps | Explicitly stated and implicitly discovered research gaps, unaddressed edge cases, and missing benchmarks. |
| 7 | Future Directions | Strategic roadmap items, recommended theoretical extensions, and concrete algorithmic scaling proposals. |
| 8 | Novel Paper Suggestions | Actionable novel research paper titles, problem formulations, and thesis project concepts for researchers. |
+-----------------------------------------------------------------------------------------------+
| RESEARCHPAPERRAG ARCHITECTURE MAP |
+-----------------------------------------------------------------------------------------------+
[ CLIENT LAYER ] - React 18 Single Page Application (Vite + Tailwind CSS + Framer Motion)
+---------------------------------------------------------------------------------------------+
| - Multi-Source Search Bar (arXiv, Crossref, Semantic Scholar, OpenAlex, PubMed, DOAJ) |
| - 3-Year Date Filter (Strict 2024–2026 publication validation) |
| - Max 5-Paper Selection checklist & PDF drag-and-drop file dropzone |
| - Internal Model Selector Dropdown & "Save Model" Action Bar |
| - 8-Layer Structured Report Card Renderer |
| - Compact Inline Citation Badge Normalizer ([Paper Title, Year, Section]) |
| - Multi-Format Exporters (Print .pdf, Raw .md, Plain .txt) |
+---------------------------------------------------------------------------------------------+
|
REST API Requests (/api/search, /api/analyze, /api/chat)
|
v
[ BACKEND LAYER ] - FastAPI / Gunicorn WSGI Engine (Python 3.11+)
+---------------------------------------------------------------------------------------------+
| 1. Concurrent Academic Search Fetcher |
| - Parallel HTTP requests via aiohttp to 6 academic search provider APIs |
| - Abstract extraction, DOI mapping, and publication year filtering (>= 2024) |
| |
| 2. PDF Ingestion & Document Processing |
| - PyPDF Loader -> Text Extraction -> RecursiveCharacterTextSplitter (chunk_size=1000) |
| |
| 3. Local CPU Zero-Key Vector Engine |
| - HuggingFace sentence-transformers/all-MiniLM-L6-v2 (100% CPU local embeddings) |
| - FAISS / Chroma Vector Stores (Per-session UUID directory isolation) |
+---------------------------------------------------------------------------------------------+
|
Vector Context Retrieval
|
v
[ SYNTHESIS ENGINE ] - 7-Stage Multi-Agent Orchestration
+---------------------------------------------------------------------------------------------+
| - [Stage 1: Extractor] -> Similarity search retrieval from session vector store |
| - [Stage 2: Analyzer] -> Context relevance scoring & chunk deduplication |
| - [Stage 3: Preprocess] -> Text cleaning & whitespace normalization |
| - [Stage 4: Optimizer] -> Token window trimming to fit target LLM context budget |
| - [Stage 5: Synthesizer] -> 8-Layer Prompt Construction & comparative matrix generation |
| - [Stage 6: Validator] -> Fact grounding & uncertainty marker verification |
| - [Stage 7: Assembler] -> Response packaging with citation badges and telemetry |
+---------------------------------------------------------------------------------------------+
|
Multi-Provider Model API Calls
|
v
[ AI PROVIDER FAILOVER LAYER ] - 6 AI Cloud Platforms / 27+ Pre-configured Models
+---------------------------------------------------------------------------------------------+
| - Groq LPU (Primary Default): llama-3.3-70b-versatile, deepseek-r1-distill-llama-70b |
| - Google Gemini: gemini-1.5-flash, gemini-1.5-pro, gemini-2.0-flash-exp |
| - OpenRouter Free: meta-llama/llama-3.3-70b-instruct:free, deepseek/deepseek-r1:free |
| - Cerebras Engine: llama3.3-70b, llama3.1-8b (2000+ tokens/sec inference) |
| - SambaNova Cloud: Meta-Llama-3.3-70B-Instruct, DeepSeek-R1-Distill-Llama-70B |
| - Hugging Face: Qwen/Qwen2.5-Coder-32B-Instruct, mistralai/Mistral-7B-Instruct-v0.3 |
+---------------------------------------------------------------------------------------------+
- Renders a responsive, modern user interface built with Tailwind CSS, Framer Motion animations, and Lucide React icons.
- Handles real-time search filtering, paper selection state management, model selector dropdown overlay, report rendering, and file export formatting.
- Asynchronous Python web service running FastAPI routed through Uvicorn or multi-worker Gunicorn.
- Exposes REST endpoints for academic literature search (/api/search), RAG gap analysis (/api/analyze), follow-up chat (/api/chat), PDF upload (/upload), and model status (/models).
- Uses sentence-transformers/all-MiniLM-L6-v2 running 100% locally on CPU to generate 384-dimensional dense vector embeddings.
- Eliminates third-party embedding API costs, quota errors, and external embedding key requirements.
- Stores vectorized paper chunks in local persistent indexes.
- Session isolation ensures each browser session gets an isolated vector store directory (faiss_index/sessions/<session_id>/), automatically cleaned up after 3 days.
ResearchPaperRAG requires at least one free provider API key to perform LLM synthesis. Gemini and Groq are configured as recommended defaults. All providers support multi-key pooling: pass single keys or comma-separated lists (KEY_1,KEY_2,KEY_3) to automatically rotate keys on 429 rate limits.
- Available Models: gemini-3.6-flash, gemini-3.5-flash-lite, gemini-2.5-flash, gemini-2.5-pro.
- How to Obtain & Multi-Key Support:
- Visit Google AI Studio.
- Click Get API Key -> Create API key.
- Set in
backend/.env:GOOGLE_API_KEY=AIzaSy...(or comma-separated:GOOGLE_API_KEY=AIza_key1,AIza_key2).
- Available Models: llama-3.3-70b-versatile, llama-3.1-8b-instant, qwen/qwen3.6-27b, mixtral-8x7b-32768, deepseek-r1-distill-llama-70b, openai/gpt-oss-120b.
- How to Obtain & Multi-Key Support:
- Visit Groq Console.
- Sign in with Google/GitHub and navigate to API Keys.
- Click Create API Key and copy your key.
- Set in
backend/.env:GROQ_API_KEY=gsk_key1(or comma-separated for multi-key rotation:GROQ_API_KEY=gsk_key1,gsk_key2,gsk_key3).
- Available Models: openrouter/free, google/gemini-3.1-flash-lite:free, qwen/qwen3.5-27b:free, meta-llama/llama-3.3-70b-instruct:free.
- How to Obtain & Multi-Key Support:
- Visit OpenRouter Keys.
- Click Create Key and copy.
- Set in
backend/.env:OPENROUTER_API_KEY=sk-or-v1-...(or comma-separated:OPENROUTER_API_KEY=sk-or-1,sk-or-2).
- Available Models: llama3.3-70b, llama3.1-8b, gpt-oss-120b.
- How to Obtain: Visit Cerebras Cloud Console and create key(s). Accepts single or comma-separated keys.
- Available Models: Meta-Llama-3.3-70B-Instruct, DeepSeek-R1-Distill-Llama-70B.
- How to Obtain: Visit SambaNova Cloud and generate key(s). Accepts single or comma-separated keys.
- Available Models: google/gemma-4-31b-it, Qwen/Qwen3.5-27B, meta-llama/Meta-Llama-3-8B-Instruct.
- How to Obtain: Visit Hugging Face Tokens and generate token(s). Accepts single or comma-separated tokens.
# SERVER & ENVIRONMENT SETTINGS
HOST=0.0.0.0
PORT=8000
ENVIRONMENT=production
CORS_ORIGINS=*
# DEFAULT PROVIDER & MODEL SELECTION
DEFAULT_PROVIDER=gemini
DEFAULT_MODEL=gemini-3.6-flash
# AI PROVIDER API KEYS (Configure single key or comma-separated key pools)
GOOGLE_API_KEY=your_google_gemini_api_key_here
GROQ_API_KEY=key_1,key_2,key_3
OPENROUTER_API_KEY=your_openrouter_api_key_here
CEREBRAS_API_KEY=your_cerebras_api_key_here
SAMBANOVA_API_KEY=your_sambanova_api_key_here
HF_API_KEY=your_huggingface_token_here
# VECTOR STORE & STORAGE CONFIGURATION
MAX_FILE_SIZE=52428800
FAISS_PERSIST_DIR=faiss_index
MAX_VECTOR_SESSIONS=64
FAISS_SESSION_MAX_AGE_DAYS=3
# RATE LIMITING & SECURITY
RATE_LIMIT_UPLOAD_PER_MINUTE=8
RATE_LIMIT_ASK_PER_MINUTE=90VITE_API_BASE_URL=http://127.0.0.1:8000Follow these steps to run ResearchPaperRAG natively on your local development machine.
- Python: 3.11 or 3.12 (Python 3.14 compatible)
- Node.js: Node 18+ & npm 9+
- Git
git clone https://github.com/AishikTokdar/ResearchPaperRAG.git
cd ResearchPaperRAG# Create Python virtual environment
python -m venv .venv
# Activate virtual environment
# Windows PowerShell:
.venv\Scripts\activate
# Linux / macOS:
source .venv/bin/activate
# Upgrade pip and install backend requirements
pip install --upgrade pip
pip install -r backend/requirements.txt
# Configure backend environment (.env)
cp backend/.env.example backend/.envChoose one of the following commands:
cd backend
python -m uvicorn app.main:app --host 127.0.0.1 --port 8000 --reloadpython -m uvicorn backend.app.main:app --host 127.0.0.1 --port 8000 --reloadThe backend service will start at http://127.0.0.1:8000.
cd frontend
# Install Node dependencies
npm install
# Create frontend environment configuration (.env)
cp .env.example .env
# Launch Vite development server
npm run devThe Vite dev server will start at http://localhost:5173.
- React SPA Frontend: http://localhost:5173 (or http://localhost:5173/chat)
- Interactive Swagger Sandbox: http://127.0.0.1:8000/docs
- ReDoc Technical View: http://127.0.0.1:8000/redoc
- API Health Check: http://127.0.0.1:8000/health
Note on Custom Localhost Backend Ports (e.g. 7860): If running the backend locally on a different port (e.g. 7860), you can append
?port=7860to your browser URL (e.g.http://localhost:5173/?port=7860) or configureVITE_API_BASE_URLinfrontend/.env.
For production deployments on Linux VPS, EC2, or Docker containers, run the FastAPI backend using Gunicorn with worker processes:
cd backend
source .venv/bin/activate
# Execute Gunicorn WSGI with 4 Uvicorn workers
gunicorn app.main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000- -w 4: Runs 4 concurrent worker processes for high-concurrency request processing.
- -k uvicorn.workers.UvicornWorker: Uses Uvicorn's asynchronous worker class for FastAPI compatibility.
- --bind 0.0.0.0:8000: Binds server to port 8000 on all network interfaces.
Deploy both the React SPA frontend (bundled with Nginx API reverse proxy) and the FastAPI backend to any remote VPS server (AWS EC2, DigitalOcean, Hetzner, Coolify, Linode, etc.) with automated healthchecks:
# 1. Clone repository on your remote VPS / server
git clone https://github.com/AishikTokdar/ResearchPaperRAG.git
cd ResearchPaperRAG
# 2. Ensure backend/.env exists with your API key configuration
cp backend/.env.example backend/.env
# 3. Build and launch production containers in detached background mode
docker compose up -d --build- Web SPA Frontend (Ports 80 & 5173):
http://your-vps-ip/orhttp://your-domain.com/ - FastAPI Backend API (Port 8000):
http://your-vps-ip:8000/docs
The frontend container includes an optimized Nginx configuration that automatically proxies /api/, /health, /upload, /ask, /models, and /docs to http://backend:8000. Deploying to any public IP address or custom domain works out-of-the-box with 0 CORS issues or SSL/HTTP mixed content errors.
# Build unified root image
docker build -t researchpaperrag .
# Run container on port 7860
docker run -d \
-p 7860:7860 \
--env-file backend/.env \
--name researchpaperrag \
researchpaperragcd backend
# Build Docker image
docker build -t researchpaperrag-backend .
# Run container with environment file
docker run -d \
-p 8000:8000 \
--env-file .env \
--name researchpaperrag-backend \
researchpaperrag-backendDeploying the Python FastAPI backend on Hugging Face Spaces provides free hosting with hardware acceleration options (ZeroGPU or CPU Basic 16 GB RAM).
-
Create a New Space:
- Go to Hugging Face Spaces.
- Enter Space Name:
researchpaperrag-backend. - License: Select MIT.
- SDK: Select Gradio (or Docker).
- Hardware: Select ZeroGPU (or CPU Basic 2 vCPU 16 GB RAM free tier).
-
Upload Repository Files:
- Push or upload all contents from the
backend/folder into the root directory of your Hugging Face Space repository (app.py,requirements.txt,app/folder, etc.).
- Push or upload all contents from the
-
Configure Backend Environment Variables & Secrets: In your Hugging Face Space dashboard, navigate to Settings -> Secrets and Variables:
-
Repository Secrets (Add your API keys securely):
GROQ_API_KEY: Your Groq Cloud API key (gsk_...)GOOGLE_API_KEY: Your Google Gemini API key (AIzaSy...)OPENROUTER_API_KEY: Your OpenRouter API key (sk-or-v1-...)CEREBRAS_API_KEY: Your Cerebras Cloud API keySAMBANOVA_API_KEY: Your SambaNova Cloud API keyHF_API_KEY: Your Hugging Face user access token (hf_...)
-
Variables (Add public runtime configuration):
DEFAULT_PROVIDER:geminiDEFAULT_MODEL:gemini-3.6-flashENVIRONMENT:productionCORS_ORIGINS:*(or your deployed Vercel / Cloudflare Pages URL)
-
-
Verify Deployment & Obtain Public URL:
- Once Hugging Face finishes building the Space, the FastAPI backend will be live.
- Copy your Space's public HTTPS URL from the Space header (e.g.
https://YOUR_USERNAME-researchpaperrag-backend.hf.space). - Verify health by opening
https://YOUR_USERNAME-researchpaperrag-backend.hf.space/healthor/docsin your browser.
Deploying the React 18 SPA frontend on Vercel distributes your user interface across global edge CDNs.
-
Create Vercel Project:
- Log in to your Vercel Dashboard and click Add New... -> Project.
- Import your
ResearchPaperRAGGitHub repository.
-
Configure Build Settings:
- Framework Preset: Select Vite.
- Root Directory: Click edit and select
frontend. - Build Command:
npm run build - Output Directory:
dist
-
Configure Frontend Environment Variables: In the Vercel project deployment screen, expand Environment Variables:
- Key:
VITE_API_BASE_URL - Value:
https://YOUR_USERNAME-researchpaperrag-backend.hf.space(Replace with the Hugging Face Space URL obtained in Part A)
- Key:
-
Deploy:
- Click Deploy.
- Vercel will build the frontend assets, compile TypeScript, and publish your production site to a custom URL (e.g.
https://research-paper-rag.vercel.app).
-
Test Full-Stack Connection:
- Open your Vercel URL in your browser.
- Navigate to
/chatand execute a topic search across academic APIs. - The React app will make REST API calls to your Hugging Face Space backend, synthesize 8-layer gap reports, and run grounded follow-up chat.
Deploying the React 18 SPA frontend on Cloudflare Pages provides ultra-fast static hosting on Cloudflare's global edge network.
-
Connect Repository to Cloudflare Pages:
- Log in to your Cloudflare Dashboard and navigate to Workers & Pages -> Create application -> Pages tab -> Connect to Git.
- Select your GitHub account and choose the
ResearchPaperRAGrepository.
-
Configure Build & Deployment Settings:
- Project Name:
researchpaperrag - Production Branch:
main - Framework Preset: Select Vite (or None).
- Expand Root Directory (advanced):
- Path:
frontend
- Path:
- Build Command:
npm run build - Build Output Directory:
dist
- Project Name:
-
Configure Environment Variables: Expand Environment Variables (advanced):
- Variable Name:
VITE_API_BASE_URL - Value:
https://YOUR_USERNAME-researchpaperrag-backend.hf.space(Replace with your live backend API URL from Part A) - Ensure the variable is added under Production (and Preview if deploying PRs).
- Variable Name:
-
Deploy:
- Click Save and Deploy.
- Cloudflare Pages will compile TypeScript, bundle frontend assets with Vite, and publish your production site to a custom URL (e.g.
https://researchpaperrag.pages.dev).
-
Troubleshooting Environment Variables on Cloudflare Pages:
- Cloudflare Pages bakes environment variables into static JS bundles at build time. If you add or update
VITE_API_BASE_URLafter creating the project, go to Deployments ->...(three dots) -> Retry deployment for the changes to take effect. - SPA client-side routing (e.g. direct page refresh on
/chator/dashboard) is automatically enabled via the bundledfrontend/public/_redirectsfile (/* /index.html 200).
- Cloudflare Pages bakes environment variables into static JS bundles at build time. If you add or update
ResearchPaperRAG provides interactive OpenAPI / Swagger UI documentation at /docs.
| Method | Endpoint | Description | Request Payload / Params |
|---|---|---|---|
| GET | /health | Live backend health check status | None |
| GET | /api/search | Concurrent search across 6 academic APIs | query (string), limit (int) |
| POST | /api/analyze | 8-Layer structured RAG gap synthesis | topic (string), provider (string), model (string) |
| POST | /api/chat | Grounded follow-up Q&A chat | question (string), provider (string), model (string) |
| POST | /upload | Upload custom PDF documents (<= 50 MB) | files (multipart/form-data) |
| GET | /models | List pre-configured models & credential status | None |
| GET | /pipeline-info | 7-stage processing pipeline telemetry | None |
| GET | /docs | Interactive Swagger UI testing sandbox | None |
- Resolution: Updated use-health.ts includes a 3-consecutive-failure threshold and a 12-second timeout allowance. Ensure you are running the latest frontend build.
- Resolution: Open /chat, select your model from the "Select AI Model" dropdown, and click "Save Model". Alternatively, edit DEFAULT_MODEL in backend/.env.
- Resolution: No. All 6 search APIs (arXiv, Crossref, Semantic Scholar, OpenAlex, PubMed, DOAJ) are 100% free and open-access. API keys are only required for the LLM synthesis layer.
| Layer | Component | Technologies |
|---|---|---|
| Frontend | User Interface | React 18, TypeScript 5.4, Vite, Tailwind CSS, Framer Motion, Lucide Icons, Sonner |
| Backend | API Web Service | FastAPI, Uvicorn, Gunicorn WSGI, Pydantic v2, Python 3.11+ |
| Vector Engine | Search & Storage | FAISS, Chroma, sentence-transformers/all-MiniLM-L6-v2 (Local CPU) |
| Academic APIs | Search Providers | arXiv API, Crossref REST API, Semantic Scholar Graph API, OpenAlex API, PubMed Entrez, DOAJ |
| AI Layer | LLM Providers | Groq LPU, Google Gemini, OpenRouter Free, Cerebras, SambaNova, Hugging Face |
This project is open-source and released under the MIT License.
Created and maintained by Aishik Tokdar (@AishikTokdar). Feel free to inspect, extend, and deploy ResearchPaperRAG for academic, personal, or commercial research workflows!