A curated list of Retrieval-Augmented Generation (RAG) resources for people shipping RAG to production.
- Which RAG architecture do you need?
- Frameworks & Orchestration
- Vector Databases & Search Backends
- Chunking Strategies
- Retrieval Techniques
- Reranking
- Evaluation & Observability
- Cost & Latency Optimization
- Failure Modes & Anti-Patterns
- Production Case Studies
- Papers (the essentials only)
- Courses & Deep Dives
- Communities
- Contributing
Don't start by picking a vector DB. Start by answering: what's your document volume, latency budget, update frequency, and query complexity?
👉 Full decision guide with a flowchart →
Quick gut-check:
| Your situation | Likely starting point |
|---|---|
| < 10k docs, prototyping | SQLite + sqlite-vec, or Chroma. No infra to manage. |
| Need hybrid (keyword + semantic) search on day one | Elasticsearch/OpenSearch with dense_vector, or Weaviate |
| Docs update every few minutes, need freshness | Avoid heavy index-rebuild pipelines; look at incremental-upsert-friendly stores (Qdrant, Weaviate, pgvector) |
| Regulatory/data-residency constraints | Self-hosted: pgvector, Qdrant, Milvus, Weaviate (OSS) |
| Already on Postgres, want fewer moving parts | pgvector or pgvectorscale |
| Multi-tenant SaaS product | Look hard at per-tenant filtering performance, not just recall — most benchmarks ignore this |
Full feature/production-readiness comparison: docs/comparison-table.md
| Project | Best for | Watch out for |
|---|---|---|
| LangChain | Fast prototyping, huge integration surface | Abstraction overhead; debugging chains in prod is painful without LangSmith |
| LlamaIndex | Data-indexing-heavy pipelines, structured retrieval | Two major API eras (legacy vs. new) cause tutorial rot |
| Haystack | Teams that want explicit, inspectable pipelines | Steeper learning curve than LangChain |
| DSPy | Optimizing prompts/retrieval jointly via compiled programs | Different mental model — not a drop-in RAG framework |
| txtai | Lightweight, embedded, all-in-one (index + LLM workflow) | Smaller ecosystem/community than LangChain |
| Verba | Weaviate-native, good reference implementation | Tied to Weaviate |
| R2R | "RAG-as-a-service" with built-in eval/observability | Newer, smaller community, verify maturity for your SLA |
Rule of thumb: if your team can't explain what happens between "user query" and "context passed to LLM" without opening framework source code, you'll have a bad time debugging it in production. Prefer frameworks with visible, inspectable pipelines over "magic" for anything customer-facing.
| Project | Model | Strong point |
|---|---|---|
| pgvector | Postgres extension | Zero new infra if you already run Postgres |
| Qdrant | Standalone, Rust | Filtering performance, good docs on quantization |
| Weaviate | Standalone, Go | Hybrid search (BM25 + vector) out of the box |
| Milvus | Standalone, distributed | Scale (billions of vectors), more ops overhead |
| Chroma | Embedded/standalone | Easiest local dev experience |
| OpenSearch k-NN | Standalone | If you already run ELK/OpenSearch for logs/search |
| Vespa | Standalone | Best-in-class hybrid ranking, steep learning curve |
Before benchmarking on recall alone, check: p99 latency under concurrent writes, filtered-search performance (most demos only benchmark unfiltered ANN), and cost of the quantization/compression setting you'd actually use in prod (not the marketing benchmark's setting).
Chunking is the most underrated lever in RAG quality — and the least glamorous, which is why it's underinvested in.
- Fixed-size with overlap — simplest, baseline, breaks semantic units mid-sentence
- Recursive character/token splitting (LangChain's
RecursiveCharacterTextSplitter) — respects paragraph/sentence boundaries - Semantic chunking (split on embedding-distance discontinuities) — better coherence, extra embedding cost per doc
- Document-structure-aware chunking (markdown headers, HTML tags, PDF layout) — best quality for structured docs, most engineering effort
- Late chunking (Jina AI, 2024) — embed the full document first, pool afterward, preserves cross-chunk context
- Proposition-based chunking — split into atomic factual statements; higher precision, higher indexing cost
Reality check: most teams over-invest in fancy chunking before fixing their eval loop. Get evaluation working first (see below), then iterate on chunking with a way to measure whether it actually helped.
- Dense retrieval — embedding similarity search (the default)
- Sparse/lexical retrieval — BM25, SPLADE; still wins on exact terms, IDs, code, acronyms
- Hybrid retrieval — combine dense + sparse (typically via Reciprocal Rank Fusion); usually beats either alone
- HyDE (Hypothetical Document Embeddings) — LLM generates a hypothetical answer, embeds that for retrieval
- Multi-query retrieval — LLM rewrites the query into several variants, union the results
- Query routing — classify the query first, send to different retrieval strategies/indexes
- Parent-document / small-to-big retrieval — retrieve small precise chunks, return surrounding larger context to the LLM
- GraphRAG (Microsoft, 2024) — build a knowledge graph, retrieve via graph traversal + community summaries; strong on multi-hop/"global" questions, expensive to build and maintain
- Agentic/iterative retrieval — LLM decides whether to retrieve again based on what it's seen so far
Trap to avoid: adding techniques additively without measuring incremental value. Hybrid search, HyDE, and multi-query all add latency and cost — each needs to pay for itself against your eval set.
Retrieval gets you candidates; reranking gets you the right candidates in the right order before they hit your context window.
| Option | Notes |
|---|---|
| Cohere Rerank | Hosted, strong quality, adds network hop latency |
| BGE reranker | Open-source, self-hostable, good quality/cost tradeoff |
| ColBERT | Late-interaction — better than bi-encoders, cheaper than cross-encoders |
Cross-encoder rerankers (e.g. ms-marco-MiniLM) |
Highest quality, highest latency — rerank a small candidate set only (top 20-50) |
Rerank a shortlist (retrieve 50-100, rerank down to 5-10) — reranking your entire candidate pool defeats the purpose and the latency budget.
This is the section most awesome-lists skip, and it's the one that actually determines whether you can improve your system without breaking it.
- RAGAS — RAG-specific metrics: faithfulness, answer relevance, context precision/recall
- TruLens — tracing + evaluation, feedback functions
- Arize Phoenix — open-source LLM observability, RAG-specific traces
- LangSmith — tracing/eval, tightest fit if you're on LangChain
- DeepEval — pytest-style LLM eval framework
- Giskard — testing/vulnerability scanning for LLM & RAG pipelines
Minimum viable eval setup before you ship anything:
- A frozen set of real (or realistic) question/answer pairs with known-good context
- Automated retrieval metrics (context precision/recall, hit rate@k) run on every change
- A faithfulness/hallucination check on generated answers, not just retrieval quality
- A human-in-the-loop spot-check sample, because automated LLM-judged metrics have their own blind spots
More detail on common eval mistakes: docs/failure-modes.md
- Embedding cost: batch embedding calls, cache embeddings for unchanged content, consider smaller/self-hosted embedding models (bge-small, nomic-embed) once you've validated quality doesn't regress
- Retrieval latency: quantization (scalar/product/binary) trades a small recall hit for large memory/latency wins — usually worth it past a few million vectors
- Reranking latency: rerank a shortlist, not the full candidate set (see above); consider ColBERT-style late interaction instead of full cross-encoders if p99 latency matters
- Context length costs: bigger context ≠ better answers; every unnecessary token in context is cost and a chance for the model to get distracted ("lost in the middle" effect) — retrieve less, retrieve better
- Caching: cache at the query level (semantic cache) and the embedding level; a surprising fraction of production traffic is near-duplicate queries
- Prompt caching (if your LLM provider supports it) for static system prompts/instructions that don't change per-request
The single most useful section for practitioners. Full writeup: docs/failure-modes.md
Quick preview of what's covered there:
- "Retrieval looks fine, answers are still wrong" — usually a faithfulness/generation problem, not a retrieval problem, and teams waste weeks tuning the wrong stage
- Silent embedding-model/index mismatches after a model upgrade
- Chunking that splits tables, code blocks, or numbered lists — one of the highest-frequency real-world bugs
- Over-reliance on cosine similarity thresholds as a relevance gate
- Evaluation sets that quietly leak into training or few-shot prompts
- "Lost in the middle": stuffing more context to fix a problem that's actually a ranking problem
Real write-ups from teams running RAG at scale (as opposed to demos):
- Notion — building and scaling a knowledge-base RAG system
- GitHub Copilot — retrieval-augmented code completion
- Elastic — hybrid search and RAG in production
- Pinecone customer case studies
- Anthropic — contextual retrieval
(Company blogs move/rename URLs — if a link is stale, please open a PR with the current one.)
Kept short on purpose — this isn't a paper-tracking list.
- Lewis et al., 2020 — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (the original RAG paper)
- Karpukhin et al., 2020 — Dense Passage Retrieval for Open-Domain Question Answering
- Gao et al., 2022 — Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE)
- Khattab & Zaharia, 2020 — ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction
- Edge et al., 2024 — From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Liu et al., 2023 — Lost in the Middle: How Language Models Use Long Contexts
- DeepLearning.AI — Building and Evaluating Advanced RAG Applications
- Pinecone Learning Center
- Weaviate Academy
- Hamel Husain — Evals blog series
Contributions welcome — see CONTRIBUTING.md. The bar for a new entry: it should help someone make a production decision, not just prove the list is exhaustive. Link rot, deprecated projects, and "add my repo with no context" PRs will be closed.
CC0 1.0 — public domain, use freely.