Local-first RAG. Eighteen packages. One facade.
An asynchronous, modular RAG platform and reusable Python SDK — from OCR to citations in one process.
Quickstart · Packages · Docs · Examples · Benchmarks · Contribute
rag-aio is a uv workspace monorepo of independently testable, independently deployable
packages that compose into a full Retrieval-Augmented Generation pipeline — from document
ingestion and OCR through chunking, embedding, hybrid retrieval, reranking, context
engineering, and provider-agnostic LLM generation — with observability, caching,
evaluation, bulk ingestion, a benchmark suite, and a Streamlit management console built in.
Why rag-aio?
| Common RAG-platform problem | rag-aio approach |
|---|---|
| Vendor lock-in on models and stores | Protocol-based contracts; OpenAI-compatible providers, Qdrant/sqlite/InMemory |
| "Works on my machine" demos | Fully-offline mock profile; the quickstart runs with zero services and zero API keys |
| Opaque pipelines | Stage timings on every answer, citation provenance, explainable retrieval and rerank |
| Heavy frameworks, heavy imports | Lazy loading: import rag_aio stays dependency-light; heavy backends load on demand |
| Secrets sprawl | Secrets by reference (env-var names), never values; redacted in logs, traces, and UI |
| Slow bulk onboarding | rag-mass-inject with bounded queues, per-stage concurrency, checkpoints, dead letters |
Everything ships in a single PyPI project: one rag-aio wheel contains all 18
packages' code. The base install pulls only light, pure-Python dependencies;
heavy backends are opt-in via extras:
pip install rag-aio # all packages, mock/light backends
pip install "rag-aio[all]" # + every optional backend
pip install "rag-aio[qdrant,fastembed]" # the full local RAG stack
pip install "rag-aio[streamlit]" # management console UIAvailable extras: qdrant, fastembed, sentence-transformers, faiss,
postgres, pgvector, redis, docling, msg, rtf, pydantic-ai,
streamlit, and the all catch-all. Without them, the fully-offline mock
profile still runs end to end — the quickstart below needs nothing extra.
For development, clone and sync the uv workspace instead:
uv syncThe fully offline path — no model server, no vector database, no network:
import asyncio
from rag_aio import RAG
async def main() -> None:
rag = RAG.from_config("config.toml")
await rag.ingest_directory("./documents")
result = await rag.ask("What are the authentication requirements?")
print(result.answer)
print(result.citations)
asyncio.run(main())with a two-key config.toml that selects the deterministic mock profile
(examples/quickstart_config.toml):
[embedder]
backend = "mock"
[pipeline.embedding]
policy = "mock"The same code runs against a real local stack by pointing the config at FastEmbed embeddings, Qdrant (embedded local mode), SQLite, and an OpenAI-compatible server such as LM Studio:
[generation]
provider = "lm_studio"
base_url = "http://localhost:1234/v1"Defaults are optimized for a local high-performance setup: PyMuPDF for fast PDF extraction, token-aware chunking, FastEmbed embeddings, Qdrant (embedded local mode) for vectors, BM25 + dense hybrid retrieval with RRF, CrossEncoder-style reranking on a small candidate set, and OpenAI-compatible generation (LM Studio, vLLM, Ollama, or any compatible server).
uv run rag-aio ingest ./documents --config config.toml # bulk ingest
uv run rag-aio ask "What are the auth requirements?" # one-shot question
uv run rag-aio serve # FastAPI service (/docs for OpenAPI)
uv run streamlit run packages/rag-ui/src/rag_ui/app.py # management consoleSee docs/api/README.md for the full Python, CLI, and HTTP API reference, and docs/configuration/README.md for every setting, default, and precedence rule.
Eighteen packages in six layers — each independently installable, each with its own README:
| Package | Layer | Responsibility |
|---|---|---|
| rag-core | Infrastructure | Canonical domain models and Protocol contracts (dependency-light) |
| rag-observe | Infrastructure | OpenTelemetry instrumentation, structured logging, redaction |
| rag-cache | Infrastructure | Multi-level caching (memory/disk/sqlite, redis extra) |
| rag-db-handler | Infrastructure | Persistence abstractions: SQL, document, KV, vector stores |
| rag-doc-handler | Processing | Document loading/parsing: PDF, HTML, web/sitemap, Office, email, Markdown |
| rag-ocr | Processing | Hybrid mechanical + semantic OCR with provenance-preserving regions |
| rag-embedder | Processing | Chunking, tokenization, embedding (FastEmbed local or OpenAI-compatible remote), indexing orchestration |
| rag-retrieval | Intelligence | Dense/sparse/hybrid retrieval, fusion, filters, explainability |
| rag-rerank | Intelligence | Second-stage relevance optimization |
| rag-query | Intelligence | Query normalization, rewriting, expansion, routing |
| rag-context | Intelligence | Context engineering: budgets, dedup, expansion, citations |
| rag-generation | Intelligence | Provider-agnostic generation with streaming and structured output |
| rag-llm-provider | Intelligence | Provider/model registry, capabilities, fallback chains |
| rag-orchestrator | Service | Pipeline management, structured concurrency, FastAPI service |
| rag-mass-inject | Operations | High-throughput bulk ingestion with checkpoints |
| rag-eval | Operations | Retrieval and generation quality metrics, reproducible runs |
| rag-ui | Operations | Streamlit management and experimentation console |
| rag-aio | Facade | RAG facade, Typer CLI, FastAPI service composition |
Everything is cross-linked from here; each package README links back to this file.
| Section | Contents |
|---|---|
| docs/architecture/ | As-built deep dive: layer diagram, ingestion/query paths, contract matrix, jobs |
| docs/modules/ | Per-package index: layer, responsibility, dependencies, extension points |
| docs/api/ | Python facade, CLI, and HTTP API reference (endpoints, SSE, versioning) |
| docs/configuration/ | Full RAGConfig reference: fields, defaults, TOML examples, secrets, precedence |
| docs/deployment/ | Topologies (embedded / workstation / scaled services) and run commands |
| docs/testing/ | Test strategy: markers, fixtures/fakes, CI behavior, e2e auto-skip |
| docs/performance/ | Benchmark methodology and measured results |
| docs/development/ | Setup, quality gates, conventions, ADR index, security workflow |
| docs/development/security.md | Security posture and Mimosa scan results |
| docs/development/releasing.md | Release process (versioning, build, PyPI trusted publishing) |
| docs/development/final-report.md | Project-closing report: modules, config, providers, results, topologies |
| docs/development/implementation-guide.md | Wave-by-wave build plan (historical, all waves done) |
| architecture.md · design.md | System architecture and design principles |
| adr/ | Architecture decision records (ADR-0001 … ADR-0008) |
| DECISIONS.md · CHANGELOG.md | Decision log and release history |
Runnable, verified scripts under examples/ — each prints what it demonstrates and exits cleanly when a model server is unreachable:
- quickstart.py — the README flow, fully offline
- stages/ — one script per pipeline stage (ingest, retrieve, rerank, context, generate)
- fully_local.py — real FastEmbed + Qdrant + SQLite + LM Studio end to end
- openai_compatible.py — any OpenAI-compatible endpoint, env-configured
web-example/ is a standalone multi-page web UI composed from the
packages above (not a workspace member, not in the release wheel): cookie-session auth
with signup/login, a ChatGPT-style streaming chat page with attachments, a document
explorer with per-user and global document pools, mass upload/injection with live job
progress, and a settings console for providers/models (chat, embedding, OCR, rerank),
retrieval mode (hybrid vs dense), and storage/DB credentials — secrets by reference.
FastAPI + Jinja2 + HTMX + Alpine, fully Python:
uv sync
uv run uvicorn web_example.main:app --reload --app-dir web-example/src
# open http://127.0.0.1:8000 — the first registered account becomes adminSee docs/modules/web-example.md for the architecture, pages, and the document-visibility model.
The pytest-based harness in benchmarks/ measures document,
embedding, retrieval (dense/hybrid), rerank, context-build, generation, and end-to-end
query performance with P50/P90/P95/P99, cold/warm splits, and RAM tracking; reports land
in gitignored benchmarks/results/. Headline numbers from the reference run: hybrid
retrieval ~65 ms, rerank ~2 ms, context build ~6 ms, full offline query ~26 ms — see
docs/performance/README.md for the full table and caveats.
uv sync # create environment from lockfile
uv run python scripts/check.py # format + lint + types + tests + benchmark units
uv run python scripts/check.py --fast # skip integration/e2e and mypyCI runs the same gates on Python 3.12/3.13 across Ubuntu and Windows; releases build all
packages via uv build --all-packages and publish to PyPI with trusted publishing.
Conventions, the ADR process, and the security-scan workflow are documented in
docs/development/README.md. Architecture decisions live in
adr/; design principles in design.md.
MIT — see LICENSE.