🤖 Hybrid RAG Project Assistant MVP

Welcome to the RAG Project Assistant—a hybrid private-cloud Retrieval-Augmented Generation (RAG) assistant built from the ground up using LangGraph v1.2+ and modern LangChain standalone integration packages.

This system represents a state-of-the-art hybrid AI architecture. It performs data parsing, text chunking, embedding generation, and vector retrieval fully locally and privately on your own computer. It only routes synthesis and evaluation queries to the cloud via the high-speed DeepSeek API, guaranteeing high security, minimal data egress, and extremely low API costs.

---

🏗️ Architecture & Core Components

graph TD
    User([User Input]) --> Streamlit[app.py Streamlit UI]
    Streamlit --> Agent[my_agent/agent.py compiled graph]
    
    subgraph AgentGraph [LangGraph Workflow]
        Start([Start]) --> Retrieve[retrieve_node]
        Retrieve --> Generate[generate_node]
        Generate --> Evaluate[evaluate_node]
        
        Evaluate -- "Fail (loop_count < 3)" --> WebSearch[web_search_node]
        WebSearch --> Generate
        Evaluate -- "Pass OR Max Loops" --> End([End])
    end

    Retrieve -.-> Chroma[(Chroma Vector DB)]
    Chroma -.-> Ollama[Local Ollama: qwen3-embedding:0.6b]
    
    WebSearch -.-> Tavily[Tavily Web Search API]
    
    Ingest[ingest.py CLI] --> Chroma
    Docs[docs/ directory] --> Ingest

Loading

---

🔍 Deep-Dive Dataflow & Node Specifications

The assistant leverages a directed acyclic state graph compiled with an in-memory MemorySaver checkpointer, preserving conversation thread context for the duration of a browser session. Below is a detailed breakdown of the execution flow:

1. Retrieve Node (retrieve_node)

• Inputs: Reads user_query from the graph state.
• Mechanism: Invokes the local vector store retriever. It leverages local Ollama running the qwen3-embedding:0.6b model to vectorize the query, performing a cosine-similarity search against local Chroma DB vectors stored in ./chroma_db/.
• Output: Retrieves the top 4 most relevant text segments and saves them to documents in the graph state.

2. Generate Node (generate_node)

• Inputs: Reads messages chat history, retrieved documents context, and any evaluation_feedback from the state.
• Mechanism:
  • Formulates a rich, grounded system prompt injecting the retrieved context.
  • If a previous candidate answer failed the evaluator audit, it automatically appends the grader's critical feedback to the prompt, instructing the model to refine its answer.
  • Calls the fast deepseek-v4-flash model with thinking disabled to ensure near-zero latency.
• Output: appends the generated AIMessage to messages and increments the loop_count state variable.

3. Evaluate Node (evaluate_node)

• Inputs: Reads the latest generated answer, retrieved documents, and the original user_query.
• Mechanism:
  • Invokes an objective LLM auditor running deepseek-v4-flash.
  • It utilizes LangChain's with_structured_output bound to a strict Pydantic schema to grade the response.
• Pydantic Grader Schema (GradeAnswer):

  class GradeAnswer(BaseModel):
      hallucination_score: str # 'yes' or 'no' (Is the answer grounded in the retrieved docs?)
      relevance_score: str     # 'yes' or 'no' (Does the answer resolve the user's query?)
      feedback: str            # Detailed critique if either score is 'no'

• Output: Sets evaluation_passed to True/False and writes the critique to evaluation_feedback in the state.

4. Web Search Fallback Node (web_search_node)

• Inputs: Executed only if evaluation_passed is False and loop_count is less than 3.
• Mechanism:
  • Takes the query and the grader's feedback to formulate an optimized search term.
  • Invokes the modern standalone TavilySearch tool (langchain-tavily) to retrieve live, external search results.
  • Automatically parses the new search result payload dictionary (results list key extraction) and merges it as a supplementary source document.
• Output: Appends the search results to documents in the state and routes control back to generate_node to perform a self-corrective rewrite loop.

---

📁 Project Directory Structure

.
├── my_agent/                 # Main agent package
│   ├── utils/                # Utilities, tools, and schemas
│   │   ├── __init__.py
│   │   ├── state.py          # Graph state schema definition (TypedDict)
│   │   ├── tools.py          # Chroma retriever & Tavily Search configurations
│   │   └── nodes.py          # State processing nodes (retrieve, generate, evaluate, search)
│   ├── __init__.py
│   └── agent.py              # Main graph setup, conditional edges, and MemorySaver compiler
├── docs/                     # Local document storage directory (Git-ignored)
│   └── .gitkeep              # Folder placeholder ensuring tracking of empty docs directory
├── .env                      # API keys and secret variables (ignored by Git)
├── .env.example              # Key configuration template
├── ingest.py                 # Document loader, segment splitter, and Chroma persistence CLI
├── langgraph.json            # LangGraph CLI configuration mapping
├── pyproject.toml            # Package configuration and dependencies managed by uv
└── README.md                 # Highly detailed project guide (this file)

---

🚀 Installation & Setup Guide

1. Prerequisites

• Ollama: Download and install Ollama. Keep the service active.
• uv: Ensure the high-performance Python package installer uv is installed.

2. Download the local Embedding Model

To generate vector embeddings locally without sending raw files to the cloud, pull the Qwen embedding model:

ollama pull qwen3-embedding:0.6b

3. Setup API Keys

Create a local .env file in the root workspace (based on .env.example):

# DeepSeek API configuration key
DEEPSEEK_API_KEY=your_deepseek_api_key_here

# Tavily Web Search API key
TAVILY_API_KEY=your_tavily_api_key_here

4. Install Dependencies

Synchronize your virtual environment and install all packages in milliseconds using uv:

uv sync

---

🛠️ Executing the Assistant

Step 1: Place and Ingest Your Private Documentation

1. Drop any local markdown files (.md) or PDF manuals (.pdf) inside the docs/ folder.

2. Run the ingestion CLI:

   uv run python ingest.py

   The script automatically parses all documents using TextLoader and the high-speed PyMuPDF library, splits them recursively into 1000-character segments, runs local embeddings, and PERSISTS the database in the local ./chroma_db folder.

   Tip: To ingest from a custom folder outside the workspace, use:

   uv run python ingest.py --docs-dir "C:\Path\To\External\Docs"

Step 2: Launch the Stock Streamlit UI

Start the web application:

uv run streamlit run app.py

Open http://localhost:8501 in your browser. As you chat, the Stock Streamlit Chat UI streams graph outputs directly to your terminal. It also presents live, collapsible execution logs for each node inside native st.expander components, giving you complete visibility into the retrieved text chunks, grader evaluation metrics, and search queries instantly!

---

🔍 Troubleshooting & FAQs

Q1: Ingest script fails with "ConnectionRefusedError: [WinError 10061]"

• Cause: Your local Ollama server is offline or is running on a non-standard port.
• Fix: Ensure Ollama is running (check your Windows system tray). You can verify it by opening http://localhost:11434 in your browser.

Q2: Streamlit shows "pydantic_core._pydantic_core.ValidationError: If using default api base, DEEPSEEK_API_KEY must be set."

• Cause: Your .env file is missing or DEEPSEEK_API_KEY is empty/incorrect.
• Fix: Make sure you have created the .env file at the root of the project containing your valid sk-... API key.

Q3: Web Search fails or returns nothing

• Cause: Missing or invalid TAVILY_API_KEY.
• Fix: Ensure your Tavily key is correctly configured inside .env. If you do not have a Tavily search key, the RAG assistant will still run completely offline/locally, but the web search fallback node will print warnings and exit.

---

🧑‍💻 Code Quality & Linter Compliance

The codebase adheres to the strictest Python standards, completely verified by Ruff:

• No Unused Imports: All redundant imports have been removed.
• PEP Compliance: Delayed, import-time environment loading statements (essential for ChatDeepSeek key validation) are fully annotated using # noqa: E402 to satisfy import-ordering guidelines.

To run the linter and verify code quality:

uv run ruff check .

Output:

All checks passed!