SmartDocAI

SmartDocAI is a local PDF question-answering application. It extracts text with PyMuPDF, groups nearby passages by embedding similarity while retaining page numbers, indexes them in Chroma, searches with BM25 and dense embeddings, reranks a small candidate set with a local cross-encoder, and asks a locally installed Ollama model to answer with page citations. A small FastAPI and vanilla JavaScript interface replaces the earlier Streamlit prototype.

This is a single-user local demonstration. It is not approved for confidential documents or safety-critical decisions. No paid API is required.

The interface serves a locally bundled Geist font under the SIL Open Font License (license); it makes no font CDN request at runtime. It also bundles Markdown, KaTeX, and Mermaid assets locally. Mathematical notation, tables, code, and diagrams in generated answers can render in the browser. Every citation can display the original PDF page, preserving figures and visual layout for inspection; the app does not infer equations or chart data from page images.

For architecture decisions, limitations, resume alignment, and interview questions, read the interview brief.

Start locally

Prerequisites: Python 3.12, uv or pip, and Ollama. The model commands download free weights once; runtime can be offline after dependencies and weights have been staged.

ollama serve

In a second terminal:

ollama pull nomic-embed-text
ollama pull llama3.2:3b
cd SmartDocAI
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python torch --index-url https://download.pytorch.org/whl/cpu
uv pip install --python .venv/bin/python -r requirements-dev.txt
.venv/bin/python -m uvicorn app:app --host 127.0.0.1 --port 8000

Open http://127.0.0.1:8000. Upload a text PDF, select one or more documents, select an installed generation model, and ask a question. Cited source labels open the locally stored PDF at the cited page. The model menu reads Ollama's locally installed model list. You can add another free local model with ollama pull <model-name> and refresh the page. Model choice affects generation; nomic-embed-text remains the embedding model for a consistent index. An index created with a different embedding model should be rebuilt.

For Docker Compose, stage model weights into its named Ollama volume before asking questions:

docker compose up --build -d
docker compose exec ollama ollama pull nomic-embed-text
docker compose exec ollama ollama pull llama3.2:3b

Then open http://127.0.0.1:8000. Compose uses an internal ollama hostname and binds both exposed ports to localhost. It does not automatically make the deployment air-gapped: build the images and stage all Python packages and model weights in the intended isolated environment before disconnecting it.

The built browser assets are committed, so normal local runtime needs neither Node nor a CDN. To rebuild them after changing render-src.js or mermaid-src.js, run npm ci && npm run build:web. Direct library licenses are under web/vendor/licenses, with additional bundled notices in mermaid.bundle.js.LEGAL.txt.

Why these components

Component	Choice and trade-off
PyMuPDF	Fast page extraction with source page numbers. Scanned pages need OCR, which this app does not include.
Semantic chunking	Paragraph and sentence units are grouped by adjacent nomic-embed-text cosine similarity, with a 900-character size cap and page boundaries. The fixed similarity threshold is a heuristic; it requires an extra embedding pass during ingestion.
nomic-embed-text	Free local dense embeddings capture related wording. Exact part numbers can still be missed, which is why BM25 is also used.
Chroma	Simple persistent local vector index. BM25 is rebuilt from the stored chunk text at startup, so there is no second document store.
BM25 + RRF	BM25 supports exact terms and dense retrieval supports paraphrases. Reciprocal Rank Fusion combines their ranks without pretending the two scores share a scale.
Local cross-encoder	Reranks only twelve fused candidates. Its weights are loaded once per process, rather than for every question; reranking costs extra CPU time.
Ollama generation	The UI prefers qwen2.5:3b when installed because it answered all eight answerable cases in the current small synthetic fixture; llama3.2:3b remains selectable and is used by the basic setup commands. This is not a general model ranking. Smaller models can still make unsupported or incomplete claims.

The application indexes all extracted chunks rather than stopping at 50, in 32-chunk batches. It rolls back a document if embedding or indexing fails and never substitutes zero vectors. This removes the old cap; it does not establish a measured 10,000-chunk throughput claim. Source passages are displayed beside answers. If the model omits citations or cites a nonexistent passage number, the API rejects that answer and asks the user to inspect the retrieved sources. Citation markers alone do not prove every claim is supported.

Verify

.venv/bin/python -m pytest -q
.venv/bin/python evaluate.py --retrieval-only
.venv/bin/python evaluate.py --model llama3.2:3b

The fixed evaluation set has ten synthetic questions (eight answerable, including two paraphrases, and two unanswerable) over two generated three-page PDFs in evaluation/cases.json. It reports Recall@5, Recall@10, MRR, citation support, exact-phrase answer correctness, abstention accuracy, p50/p95 query latency, and error rate. Retrieval and answer metrics use the eight answerable questions as their denominator; abstention uses the two unanswerable questions. These are small synthetic checks, not aerospace accuracy measurements. The exact-phrase answer check is deliberately strict and can mark a correct paraphrase wrong. Results should be reported with the model, dataset size, hardware, and whether generation was included. No benchmark score is claimed in this README.

Scope and security

• The app is intended for a trusted user on one machine. It has no accounts, access controls, OCR, or multiuser isolation.
• Uploaded PDFs and their index persist under data/ for local source viewing. Remove a document in the UI to delete its indexed chunks and stored PDF.
• Each question is retrieved and answered independently. References like “what about it?” need a fuller question; conversation-aware retrieval is not implemented.
• The application does not call a paid service. At runtime it contacts the Ollama host configured by OLLAMA_HOST and no external API; check that host and network policy before calling a deployment air-gapped.
• Earlier repository revisions tracked Ollama identity files. The current working tree stops tracking them, but public Git history may still contain them. Replace the exposed identity and review history separately; .gitignore alone cannot erase old commits.
• Never use confidential defense or aerospace material to test this public project. Use synthetic or public PDFs.