Codebase RAG — Fully Local, Offline AI Codebase Assistant

An AI assistant that indexes an entire software project and answers questions about it — architecture, auth flows, "where is X implemented", duplicate logic, dependency graphs — using only local, free, open-source components. No OpenAI, no Gemini, no Claude API, no paid vector database. Everything — parsing, embeddings, vector search, and the LLM itself — runs on your machine.

Repo (zip / local folder)
        │
        ▼
┌───────────────────┐     ┌──────────────────┐     ┌───────────────────┐
│  repo_loader.py    │────▶│  code_parser.py   │────▶│   chunker.py       │
│  walk + ignore     │     │  tree-sitter AST   │     │  size-bound chunks │
└───────────────────┘     └──────────────────┘     └─────────┬─────────┘
                                                                │
                          ┌─────────────────────────────────────┘
                          ▼
                ┌───────────────────┐        ┌────────────────────┐
                │  embedder.py       │───────▶│  vector_store.py    │  (FAISS)
                │  sentence-         │        └────────────────────┘
                │  transformers      │                 │
                └───────────────────┘                 │
                          │                             ▼
                          ▼                    ┌────────────────────┐
                ┌───────────────────┐          │      db.py          │  (SQLite)
                │  retriever.py      │◀─────────│  metadata + chunks   │
                │  hybrid search     │          └────────────────────┘
                └─────────┬─────────┘
                          ▼
                ┌───────────────────┐        ┌────────────────────┐
                │  prompt_builder.py │───────▶│  ollama_client.py    │  (local LLM)
                │  grounded prompt   │        └────────────────────┘
                └───────────────────┘                 │
                                                        ▼
                                              Answer + citations

Table of contents

• Quick start
• Tech stack (and why)
• Project structure
• How indexing works
• How retrieval works
• How prompting works (anti-hallucination)
• How tree-sitter parsing works
• How FAISS is used
• How Ollama integrates
• Database schema
• API reference
• Testing
• Supported languages
• Known limitations

---

Quick start

1. Prerequisites

• Python 3.10+
• Ollama installed locally
• ~2GB free disk (embedding model + an Ollama model)

2. Install

git clone <this-repo>
cd codebase-rag
python3 -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt

Note on tree_sitter version: requirements.txt pins tree_sitter==0.21.3. This is required — tree_sitter_languages (which bundles the prebuilt grammars for every supported language) was built against the pre-0.22 tree-sitter C API. If tree_sitter is upgraded past 0.21.x you'll hit TypeError: __init__() takes exactly 1 argument (2 given) when parsing any file. This was hit and fixed during development of this project — keep the pin.

3. Start Ollama and pull a model

ollama serve                 # usually auto-started by the Ollama app
ollama pull llama3           # or mistral / qwen2.5-coder / deepseek-coder

4. Start the backend

python run_api.py
# API now running at http://localhost:8000 (docs at /docs)

5. Start the frontend

streamlit run frontend/streamlit_app.py

6. Index a repository

In the UI sidebar: upload a .zip of your project, or give it an absolute local path. Watch the progress bar, then switch to the Chat tab and ask:

"Explain this project." "How does authentication work?" "Where is user login implemented?" "Which modules import AuthService?"

---

Tech stack (and why)

Concern	Choice	Why
API	FastAPI	Async, automatic OpenAPI docs, Pydantic validation built in
UI	Streamlit	Fast to build a functional chat/search/graph UI without a JS toolchain
Code parsing	tree-sitter (tree_sitter_languages)	One real AST per language, not regex guessing; prebuilt grammars = no compiler toolchain needed
Embeddings	sentence-transformers (bge-small-en-v1.5)	Small (~130MB), CPU-friendly, strong retrieval quality, fully offline after first download
Vector search	FAISS (IndexFlatIP)	Exact cosine search, zero external service, trivially persisted to disk
Metadata store	SQLite	Zero-config, file-based, powers exact filename/language/function search
LLM	Ollama (Llama 3 / Mistral / Qwen / DeepSeek)	Local inference server, simple REST API, swap models with one config change
Repo ingestion	GitPython + zipfile	Handles both "clone a URL" and "upload a zip" without extra services

No dependency in this stack requires a paid API key or makes an external network call at question-answering time. The only run-once network calls are: downloading the sentence-transformers model weights from Hugging Face on first use, and ollama pull <model> downloading model weights. Both are cached locally afterward — no further internet access is needed.

---

Project structure

codebase-rag/
├── backend/
│   ├── api/            FastAPI app + all HTTP routes (main.py)
│   ├── config/          settings.py — single source of config, env-driven
│   ├── database/        db.py — SQLite schema + all metadata CRUD
│   ├── embeddings/       embedder.py (sentence-transformers), vector_store.py (FAISS)
│   ├── llm/              ollama_client.py, prompt_builder.py
│   ├── models/           schemas.py — CodeChunk, ParsedFile, all API request/response models
│   ├── parser/           language_config.py, code_parser.py (tree-sitter), chunker.py
│   ├── retrieval/        retriever.py — hybrid vector + keyword search
│   ├── services/         repo_loader.py, indexing_service.py, qa_service.py, graph_service.py
│   └── utils/            logger.py, hashing.py
├── frontend/
│   └── streamlit_app.py  Full UI: upload, chat, search, stats, dependency graph
├── data/                 (gitignored) extracted repos, SQLite DB, FAISS index
├── tests/                 pytest suite covering parser, chunker, db, and full pipeline
├── docs/                  Deep-dive docs: architecture, API, schema, setup
├── requirements.txt
├── .env.example
├── pytest.ini
└── run_api.py             `python run_api.py` to start the backend

Every backend module is independently unit-testable and has no circular dependency: parser doesn't know about embeddings, embeddings doesn't know about database, etc. services/ is the only layer that wires multiple modules together, and api/ is the only layer that knows about HTTP.

---

How indexing works

1. repo_loader.py gets the repository onto local disk (zip extraction with zip-slip protection, a local path, or a shallow git clone) and walks its file tree, skipping node_modules, .git, venv, build, dist, __pycache__, binaries, and anything over 2MB (configurable in settings.py).
2. For each remaining file, code_parser.py detects the language by extension and runs it through tree-sitter, walking the AST to extract every function, method, class, and import statement with exact line numbers (see How tree-sitter parsing works).
3. chunker.py turns those entities into final CodeChunk objects: splitting anything over 200 lines into sub-chunks (so one giant class doesn't become one diluted embedding), adding a dedicated import-block chunk per file, and falling back to fixed 120-line windows for files with no extractable entities (e.g. YAML, plain text, or a language without a grammar) so their content is still searchable.
4. embedder.py batch-embeds each chunk's text (file path + type + name + docstring + code, so even a one-line getter carries identifying context) into a 384-dim vector.
5. vector_store.py appends those vectors to a shared FAISS index and returns the position assigned to each one.
6. db.py stores each chunk's full metadata (file, language, line numbers, docstring, code) in SQLite, along with the FAISS position — this is the link between the two stores.
7. Progress (files_indexed / files_total) is written to the repos table throughout, so the UI can poll /repos/{id}/status and show a progress bar.

Indexing is idempotent: chunk IDs are a deterministic hash of (repo_id, file_path, start_line, end_line, name), so re-indexing an unchanged file overwrites the same SQLite row rather than duplicating it. Re-indexing a whole repo first purges its old FAISS vectors (vector_store.remove_repo) so deleted/renamed files don't leave stale vectors behind.

How retrieval works

retriever.py implements hybrid retrieval:

• Vector search: the question is embedded with the same model (using bge's query-instruction prefix) and searched against FAISS for semantically similar chunks.
• Keyword/identifier search: the question is scanned for identifier-shaped tokens (e.g. UserService, handle_upload) and those are looked up directly against chunk names in SQLite. This catches the common case where a question names an exact symbol but the wording around it doesn't embed close to the code's docstring.

Results are merged, deduplicated by chunk_id, and sorted by score — keyword hits get a fixed high score since an exact name match is a strong, reliable signal. Results below a relevance floor are discarded before ever reaching the LLM (see below).

The standalone /search endpoint (used by the UI's Search tab, no LLM involved) does the same hybrid search plus explicit metadata filters: by language, chunk type (function/class/method), or filename substring.

How prompting works (anti-hallucination)

All prompt assembly lives in one file, backend/llm/prompt_builder.py, so the grounding rules are enforced uniformly:

1. Every retrieved chunk is rendered as a numbered [SOURCE N] block with its exact file path, function/class name, and line range.
2. The system prompt instructs the model to cite (SOURCE N) for every factual claim, to explicitly say when the sources don't answer the question rather than guessing, and to never invent file paths or line numbers not present in the sources.
3. If retrieval finds nothing relevant, the LLM is never called at all — qa_service.py short-circuits with an honest "not found in the indexed codebase" response. This is the single strongest anti-hallucination guarantee: an LLM can't hallucinate an answer it's never asked to generate.
4. Prior conversation turns are included for follow-up questions but are clearly labeled as "for context only, not a source of code facts" so the model doesn't treat its own earlier phrasing as newly-verified ground truth.

How tree-sitter parsing works

Rather than 11 hand-written per-language parsers, code_parser.py uses one generic AST walker driven by a declarative table (language_config.py) mapping each language to which tree-sitter node types mean "function", "class", "method", and "import":

"python": LanguageProfile(
    function_nodes={"function_definition"},
    class_nodes={"class_definition"},
    import_nodes={"import_statement", "import_from_statement"},
    docstring_capable=True,
),

The walker recursively visits every AST node; when it enters a class-like node it pushes the class name onto a stack, so any function found while that stack is non-empty is classified as a method with parent_class set — this uniformly handles Python's nested class/def, Java's class-scoped methods, C++'s member functions, and Rust's impl blocks with one piece of logic.

One subtlety fixed during development: tree-sitter grammars sometimes reuse a node type name for both a real syntax construct and an anonymous keyword token — e.g. JavaScript's function keyword literal is also typed "function", the same type used for function expressions. The walker checks node.is_named and skips anonymous tokens, which fixed a bug where every function keyword was producing a spurious <anonymous> chunk (covered by tests/test_parser.py::test_javascript_no_false_positive_on_function_keyword).

SQL has no reliable, actively-maintained tree-sitter grammar bundled in tree_sitter_languages, so it uses a dedicated statement-splitter fallback: source is split on ;, and each CREATE TABLE/VIEW/FUNCTION/... statement becomes its own named chunk.

How FAISS is used

One IndexFlatIP (exact inner-product search) is shared across all indexed repositories. Embeddings are L2-normalized before insertion, which makes inner product equivalent to cosine similarity. IndexFlatIP was chosen over an approximate index (IndexIVFFlat, HNSW, etc.) because codebases produce thousands, not millions, of chunks — exact search is fast enough and avoids any recall loss from approximation.

Because the index is shared, vector_store.py maintains a side JSON file mapping FAISS position → (repo_id, chunk_id). Searching with a repo_id filter over-fetches (top_k * 5) and discards non-matching-repo hits client-side, so results are still a full top_k within the requested repo. The index and its ID map are persisted to data/index/ after every write, so restarting the API doesn't require re-embedding anything.

Re-indexing a repo does a full index rebuild excluding that repo's old vectors (IndexFlatIP has no efficient in-place delete) — acceptable since re-indexing is an infrequent, explicit action, not a hot path.

How Ollama integrates

backend/llm/ollama_client.py is the only module in the codebase that sends a network request carrying prompt content, and it only ever talks to http://localhost:11434 (configurable via RAG_OLLAMA_HOST) — Ollama's local REST API, called directly via requests with no SDK. /api/generate is used for the main chat answer; a streaming variant is available for future token-by-token UI rendering. If Ollama isn't running, qa_service.py catches the connection error and still returns the retrieved citations with a clear message — the user gets the relevant code even if the LLM step failed, rather than a raw 500 error.

Swap models by changing RAG_OLLAMA_MODEL in .env (after ollama pull <model>) — nothing else in the codebase needs to change.

Database schema

See docs/DATABASE_SCHEMA.md for the full annotated schema. Summary:

• repos — one row per indexed repository, tracks indexing status/progress.
• files — one row per indexed file, with its language, LOC, and raw import list.
• chunks — one row per CodeChunk: the atomic retrievable unit, linked to its FAISS vector via vector_index.
• conversations / messages — chat history for follow-up-question memory.

API reference

See docs/API.md for every endpoint with example requests/responses. Interactive docs are also auto-generated by FastAPI at http://localhost:8000/docs once the server is running.

Testing

pytest tests/ -v

17 tests cover: per-language parsing correctness (including the JS keyword-token regression), oversized-chunk splitting, fixed-window fallback chunking, SQLite CRUD and vector-index linkage, and full pipeline integration tests (index a sample multi-file repo → verify chunk counts, DB/FAISS linkage, hybrid retrieval surfaces the right function, import graph resolves real cross-file dependencies, and re-indexing doesn't duplicate vectors).

Tests use a MockEmbedder fixture (deterministic hash-based vectors) so they run without downloading the real embedding model or requiring network access — every other component (tree-sitter parsing, chunking, SQLite, FAISS, retrieval, graph building) runs for real. See docs/TESTING.md for details and how to test with the real model + a live Ollama instance.

Supported languages

Python, C++, Java, JavaScript, TypeScript, Go, Rust, C#, HTML, CSS (all via tree-sitter), and SQL (via a dedicated statement-splitter, since no maintained tree-sitter SQL grammar is bundled). Any other file extension still gets indexed via fixed-window chunking, just without function/class-level structure.

Known limitations

• Import resolution is heuristic, not a real linker. graph_service.py fuzzy-matches import strings against indexed file paths by filename stem. It's accurate enough for a repository map and dependency diagram, not a substitute for a build-system-aware dependency analyzer.
• calls (call graph) extraction is not populated by the current parser. The CodeChunk schema has a calls field reserved for this; a follow-up pass could walk function bodies for call expressions per language to populate it and power a true call graph (currently the Mermaid "dependency graph" is import-based, not call-based).
• IndexFlatIP is exact search, which is a deliberate simplicity/accuracy tradeoff — for extremely large monorepos (500k+ chunks) an approximate index (HNSW) would be faster at some recall cost; swap it in vector_store.py if you hit that scale.
• The embedding model and Ollama model must be downloaded once with internet access before fully offline operation — this is unavoidable for any local-model approach and is a one-time setup step, not a runtime dependency.