RAGForChatBots — Graph-Based RAG for California Personal Income Tax

A hybrid GraphRAG chatbot over official California tax documents (Form 540 2EZ booklet, tax tables, Schedule D instructions). Answers combine two retrieval channels:

• Knowledge graph (Neo4j): at index time, claude-opus-4-8 extracts entities (forms, credits, filing statuses, thresholds) and relationships from each document chunk into a Neo4j graph. At query time, entities mentioned in the question are matched and their graph neighborhood (1–2 hops) is expanded via Cypher.
• Vector search (Chroma): markdown-aware chunks of the PDFs, retrieved by similarity. Tax tables are served exclusively by this channel. Embeddings run locally via Ollama (nomic-embed-text, free) by default, or through Voyage AI (voyage-3) when VOYAGE_API_KEY is set.

Both channels are merged into one cited context and answered by claude-haiku-4-5.

Prerequisites

• Python 3.12 + uv
• A running Neo4j instance (this project assumes the Docker container local-neo4j with ports 7474/7687)
• Ollama with the embedding model pulled: ollama pull nomic-embed-text (skip if using Voyage instead)
• Keys in .env (copy .env.example): ANTHROPIC_API_KEY, NEO4J_PASSWORD, and optionally VOYAGE_API_KEY to use Voyage embeddings instead of Ollama

The embedding model that built the index is recorded in the manifest; switching providers later requires index --force (the app refuses to query mismatched vector spaces).

Usage

uv sync

# One-time (and after document changes): build vector + graph indexes.
# Incremental — unchanged files are skipped; use --force to rebuild from scratch.
uv run main.py index [--force] [--verbose]

# Interactive Q&A. --show-sources prints the graph facts and documents used.
uv run main.py chat [--show-sources] [--verbose]

Inspect the extracted graph in the Neo4j browser at http://localhost:7474:

MATCH (e:Entity)-[r:RELATES_TO]->(n) RETURN e, r, n LIMIT 50

Architecture

main.py                     CLI (index | chat)
configs/app_config.yaml     all knobs (models, chunking, retrieval, paths)
prompt_registry/            versioned prompts (chat/v2, extraction/v1)
graphrag/
  config.py                 Pydantic config loading/validation
  prompts.py                prompt registry with {{var}} rendering
  llm.py                    model factories (sampling params gated per model)
  ingestion/                PDF→markdown chunks, sha256 manifest, LLM extraction, indexer
  graph/                    Neo4j store: schema constraints, upserts, neighborhood Cypher
  retrieval/                entity matcher (no LLM) + hybrid graph/vector retriever
  chat/                     prompt assembly (cached system prompt) + interactive loop

Key behaviors:

• Chunk ids are deterministic (<pdf-stem>:p<page>:c<seq>) and shared between Chroma and Neo4j, making indexing idempotent and citations joinable across both stores.
• The index manifest (data/db/index_manifest.json) records a sha256 per file; re-runs skip unchanged documents.
• Tax-table PDFs (*taxtable*) and mostly-tabular chunks skip graph extraction — numeric lookups are a vector-search problem.
• chat never re-indexes; it fails fast with instructions if indexes are missing.

Tests

uv run pytest                                   # unit tests, fully offline
GRAPHRAG_INTEGRATION_TESTS=1 uv run pytest      # + live Neo4j tests (WIPES Entity/Chunk data)