Skip to content

About

Retrieval-augmented chatbot that answers questions grounded in your own documents and summarises arXiv papers.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

6 Commits

Folders and files

Repository files navigation

RAG Knowledge Assistant

A retrieval-augmented chatbot that answers questions grounded in your own documents and summarises arXiv papers. Uses a 4-bit quantised Mistral 7B model running locally, with FAISS for vector search and HuggingFace embeddings.

Built as an internal knowledge tool for a research institute (~20 users), where staff needed quick answers from institutional documents without searching through files manually.

What it does

  • Question answering over documents: load any text file, the assistant chunks it, embeds it with HuggingFace, stores it in FAISS, and retrieves relevant context to answer questions. The LLM is instructed to answer only from the provided context.
  • arXiv paper summarisation: give it an arXiv ID, it downloads the PDF, reads the first pages (abstract + introduction), and returns a bullet-point summary.
  • Runs locally: the model is 4-bit quantised (NF4) so it fits on a single GPU. No API keys, no external calls (except arXiv downloads).

Architecture

                      ┌─────────────────┐
                      │   User query    │
                      └────────┬────────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                                 ▼
     Document question                  arXiv summary
              │                                 │
              ▼                                 ▼
    FAISS retriever                   Download PDF from arXiv
    (HuggingFace embeddings)                    │
              │                                 ▼
              ▼                        Split first N pages
    Top chunk as context                        │
              │                                 ▼
              └────────────────┬────────────────┘
                               │
                               ▼
                    Mistral 7B (4-bit quantised)
                               │
                               ▼
                         Answer / Summary

Setup

pip install -r requirements.txt

Requires a CUDA-capable GPU for the quantised model.

Usage

As a module:

from rag_assistant import RAGAssistant

assistant = RAGAssistant()
assistant.load_documents("path/to/your/document.txt")

answer = assistant.ask("What is the institute's policy on remote work?")
print(answer)

summary = assistant.summarise_arxiv("2405.12685")
print(summary)

Interactive mode:

python rag_assistant.py

Notebook:

The original notebook LLM.ipynb walks through each step with inline output.

Stack

Python 3.9+ · PyTorch · Mistral 7B (4-bit, bitsandbytes) · LangChain · FAISS · HuggingFace

Files

  • rag_assistant.py — main module with the RAGAssistant class
  • LLM.ipynb — interactive notebook with step-by-step walkthrough
  • requirements.txt — dependencies

Author

Ippocratis Saltas

About

Retrieval-augmented chatbot that answers questions grounded in your own documents and summarises arXiv papers.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages